12: v0.1.0 release plumbing + rc1 (#12) - #34
Merged
Merged
Conversation
Pre-alpha design document. Establishes the value proposition (LLM-drafted dbt artifacts pruned against real warehouse data), the differentiator (quality eval in the loop, reusing clauditor's grading methodology), the v0.1-v1.0 roadmap, and the design principles.
Architecture stays warehouse-agnostic; adapters plug in behind a thin sampling/profiling interface. BigQuery picked as v0.1 target for its generous sampled-read pricing and INFORMATION_SCHEMA.JOBS history. Snowflake, Databricks, Postgres, Redshift moved to v0.2+.
Sets up per-repo policy for AI tooling: synced commands/agents and non-maintainer skills stay local-only, only release-manager and review-agentskills-spec ship with the repo. Pattern mirrors clauditor. Includes the super-plan document for issue #1 (project scaffolding) so planning work survives across worktrees.
* Trim bark/beads scaffolding additions to match repo policy - CLAUDE.md: replace mandatory beads block with a short availability note. The original asserted bd as the only allowed task tracker and prescribed a fixed end-of-session push workflow, which conflicts with /super-plan and Claude Code's task tools. - AGENTS.md: same trim; keep the non-interactive shell guidance. - .claude/settings.json: track. Just bd-prime hooks for SessionStart and PreCompact — no user-specific paths. - .gitignore: ignore .claude/plugins/ (install/cache state). * Plan #1: Phases 2-3 complete (architecture review + decisions) - Architecture review surfaces 8 concerns, 0 blockers (CI permissions, action SHA-pinning, smoke test signal, hatchling wheel target, CONTRIBUTING scope, README drift, version value). - R1-R6 resolved as DEC-009..DEC-014. - Filed #13 to track the open question of long-term beads ↔ /super-plan integration; link recorded in the plan doc. - Add .beads/ to .gitignore — bark's bd init landed it in the original checkout, not the worktree; ignoring prevents accidental commits if it appears in a future checkout too. * Plan #1: Phase 4 — story breakdown (US-001..US-007) Seven stories: five implementation (foundation, lint+type, tests, CI, docs), one quality gate, one patterns-and-memory. Each story traces to specific DEC-### entries and lists explicit files, acceptance criteria, and dependencies. Validation command set: pip install -e .[dev] && ruff check . && pyright && pytest. * Plan #1: phase published (PR #14) * Plan #1: phase devolved (bd epic bd_1-scaffolding-mxk + 7 tasks) Created the bd issue graph: 1 epic, 7 tasks (US-001..US-007), 11 blocks edges. bd ready returns only US-001 — the rest are correctly blocked. Discovered (and recorded) that bd is worktree-aware via 'bd context': auto-discovers the canonical .beads/ at the main checkout from any worktree CWD. No symlink or BD_REPO_DIR env var needed; obsoletes the option-C plan. * bd_1-scaffolding-mxk.1: Add pyproject.toml + src layout + __version__ Hatchling backend with dynamic version sourced from src/signalforge/__init__.py. Explicit wheel packages declaration (DEC-011) since hatchling does not reliably auto-find src layout. Dev deps: ruff, pyright, pytest (DEC-004). Closes US-001 of #1. * bd_1-scaffolding-mxk.2: Add ruff + pyright config Per DEC-002: line-length 100, target-version py310, rule set E/F/W/I/UP/B/SIM, pyright in standard mode with src+tests roots and Python 3.11 (matches CI). Empty [tool.ruff.format] accepts defaults. Closes US-002 of #1. * bd_1-scaffolding-mxk.5: Add CONTRIBUTING.md + README v0.1 status callout Lean CONTRIBUTING (DEC-012): branch policy, local dev commands, license reminder, explicit out-of-scope note for bark/super-plan/bd (tracked in #13). README gets a sharper v0.1 status block above the quick-start (DEC-013) so readers do not try a 'pip install signalforge' that fails today. Closes US-005 of #1. * bd_1-scaffolding-mxk.3: Add tests/ + smoke test + pytest config Smoke test imports signalforge and asserts __version__ matches PEP 440 shape (DEC-010 — no assert True; the test must be capable of failing). pytest config sets testpaths=['tests'] and addopts='-ra --strict-markers' so unknown markers error out loudly instead of being silently accepted. Closes US-003 of #1. * bd_1-scaffolding-mxk.4: Add GitHub Actions CI workflow DEC-009: actions/checkout and actions/setup-python pinned to commit SHA with trailing version comment. Top-level permissions scoped to contents:read. Concurrency cancels superseded runs on the same ref. DEC-003: Python pinned to 3.11 (single version for v0.1; widen later). Triggers: PRs into dev, pushes to main. Job runs ruff check, ruff format check, pyright, pytest in series — same commands contributors run locally per CONTRIBUTING.md. Closes US-004 of #1. * bd_1-scaffolding-mxk.6: Quality gate — fix bugs from code review Quote pip install extras for zsh users (CONTRIBUTING.md, README.md): 'pip install -e .[dev]' is a glob in zsh and fails with 'no matches found'. Quoting works in both bash and zsh. Caught by QG pass 3. Fix README v0.1 prose drift: line 49 still said 'Snowflake adapter' after commit 243a709 switched v0.1 to BigQuery, contradicting the roadmap table on the next line. Caught by QG pass 4. Closes US-006 of #1. * bd_1-scaffolding-mxk.7: Add .claude/rules/* and CLAUDE.md validation cmd Capture the conventions established by issue #1 so future /super-plan runs in this repo discover them automatically: - python-build.md: Hatchling + src layout + dynamic version (DEC-011); the "wheel target packages declaration is non-negotiable" lesson. - ci-supply-chain.md: SHA-pinning, scoped permissions, concurrency cancel-in-progress (DEC-003, DEC-009). - testing-signal.md: no assert-True tests; pytest 9 needs both addopts=--strict-markers AND strict_markers=true ini setting, because the option does not propagate to getini (DEC-010, US-003). CLAUDE.md gains a 'Validation' section with the canonical four-step command shared between local dev and CI; 'Repository status' now reflects what shipped instead of "only README and LICENSE". Implemented by orchestrator after the worker hit a permission gap on .claude/ writes inside the worktree (filed for the next ralph run to include .claude/** in worker scope). Closes US-007 of #1. * Address Copilot PR #14 review comments - testing-signal.md: replace "(CLAUDE.md §1)" with explicit reference to the "Architectural commitments" heading (CLAUDE.md has no numbered §s). - test_smoke.py: rewrite the misleading comment — the test does not exercise wheel packaging (editable install goes via .pth, not the wheel target). Describe what is actually verified: idempotent import. - .github/workflows/ci.yml: quote ".[dev]" to match the documented zsh-safe convention even though Actions runs bash. - README.md: "follow-up ticket of v0.1" → "follow-up v0.1 ticket". - AGENTS.md: drop `bd dolt push` from the quick reference and replace with a comment pointing to #13 (the dolt remote URL is not wired up in this repo, so the command would fail). - plans/super/1-project-scaffolding.md: - Replace local absolute paths in Meta, Phase 2 housekeeping note, and Beads Manifest with machine-agnostic descriptions. - Update Phase from "devolved" to "implemented" (all 7 stories merged; PR ready for review). - Align the Detailed Breakdown's stated validation command with the canonical CLAUDE.md §Validation recipe (quoted extras + ruff format check). - Fix US-003 acceptance criteria: 3 tests with their actual names (was "1 test"; old name placeholder was wrong); add the pytest 9 strict-markers ini-vs-addopts caveat inline. Validation green: ruff check, ruff format --check, pyright, pytest (3 passed). ---------
* Add super plan for #2 (manifest loader) and vendored dbt research - plans/super/2-manifest-loader.md: full /super-plan output (Phases 1-4) with 17 decisions and 9 stories (US-001..US-009 incl. Quality Gate and Patterns & Memory). - docs/research/: pinned snapshots of the seven dbt-prefixed research files from clauditor's gitignored docs/temp/, so contributors outside the maintainer's machine can read the references the plan cites (DEC-006). No code yet -- this is the planning PR. Implementation lands per the beads task graph after approval. * Fill in Beads Manifest for #2 plan (Phase 7 devolve) Epic bd_1-scaffolding-28p + 9 tasks created with 10 dependency edges matching the story graph. bd ready returns only .1 (US-001). * bd_1-scaffolding-28p.1: US-001 — add Pydantic v2 + dbt-core deps and pytest markers - pydantic>=2.5,<3 as the first runtime dep (DEC-001). - dbt-core>=1.8,<2 in [dev] for fixture regeneration (DEC-009). - pytest markers unit/integration/error declared (DEC-015). * bd_1-scaffolding-28p.3: US-003 — manifest errors module (TDD) - src/signalforge/manifest/{__init__.py, errors.py}: seven-class hierarchy rooted at ManifestError, with remediation: str rendered in __str__ (DEC-013, DEC-014). - tests/manifest/test_errors.py: ≥6 fail-capable unit tests covering hierarchy, default remediation, override, and base-class catching. * bd_1-scaffolding-28p.2: US-002 — test fixtures (small × 4 schemas, medium, 5 error paths) - tests/fixtures/dbt_project_small/: 4-5 model dbt project + manifest_v9/10/11/12.json (DEC-012). - tests/fixtures/dbt_project_medium/: ~50-model synthesised project + manifest_v12.json. - tests/fixtures/error_paths/: malformed/missing_version_url/unsupported_v99/disabled_only/empty_raw_code. - tests/fixtures/regenerate.sh + README.md (DEC-009 multi-version regen recipe). * bd_1-scaffolding-28p.4: US-004 — manifest Pydantic models module (TDD) - src/signalforge/manifest/models.py: Manifest, Model, Column, Ref, Config, DependsOn — frozen + extra=ignore + populate_by_name. Validators on unique_id (must start "model.") and raw_code (strip-to-None) per DEC-016. columns_list property on Model. Nested Config per DEC-011. - tests/manifest/test_models.py: TDD tests parametrised across the four committed v9/v10/v11/v12 small manifests + extra="forbid" drift detector. * bd_1-scaffolding-28p.5: US-005 — manifest loader module (TDD) - src/signalforge/manifest/loader.py: load(), _detect_version, _canonicalise_path with .resolve()+is_relative_to() symlink hardening (DEC-007), MAX_MANIFEST_BYTES soft warning (DEC-008), manifest_path override (DEC-010), get_model/iter_models/schema_version free functions backed by lazily-built unique_id and file-path indexes. - src/signalforge/manifest/models.py: thin method wrappers on Manifest delegating to loader free functions (deferred import, no cycle). - tests/manifest/test_loader.py: >=16 fail-capable tests including the 7 Phase 2 regression tests, symlink hardening, 200MB warning, all error paths. * bd_1-scaffolding-28p.6: US-006 — public __init__.py re-exports - src/signalforge/manifest/__init__.py: re-exports load, Manifest, Model, and all error classes; declares __all__ (DEC-017). - tests/manifest/test_public_api.py: programmatic importability check, Pydantic isinstance checks, error-hierarchy preservation, and explicit guards against accidental promotion of internals. * bd_1-scaffolding-28p.7: US-007 — documentation (research index, ops guide, CONTRIBUTING) - docs/research/README.md: snapshot framing for the seven vendored dbt research files (DEC-006). - docs/manifest-loader-ops.md: memory profile table, soft 200MB warning posture (DEC-008), multi-version regen cross-link (DEC-009/DEC-012), supported-version table, error-class quick reference. - CONTRIBUTING.md: Test markers + Regenerating fixtures subsections (DEC-015 / DEC-009). - README.md: v0.1 status callout updated to reflect library-first delivery. * bd_1-scaffolding-28p.8: Quality gate — fix bugs from code-reviewer passes 1-4 Pass 2 (adversarial) surfaced two real bugs in the path-hardening logic; both are now fixed with regression tests that fail before the fix. 1. Default target/manifest.json bypassed _canonicalise_path. A symlink at target/manifest.json -> /etc/passwd would have been silently followed. Fix: route the default path through _canonicalise_path the same way the manifest_path override is handled. (DEC-007 hardening, applies uniformly.) 2. Path.resolve() raises RuntimeError on symlink cycles regardless of strict=. _canonicalise_path leaked that bare RuntimeError instead of the typed ModelPathOutsideProjectError our hierarchy promises. Fix: wrap both resolve() calls in try/except RuntimeError and re-raise with the same remediation surface. Three new fail-capable regression tests: - test_default_manifest_path_symlink_escape_is_rejected - test_symlink_loop_in_default_path_is_rejected - test_symlink_loop_in_explicit_manifest_path_is_rejected Pass 3 (test correctness) flagged one marker mis-mark: test_oversize_manifest_emits_warning was @pytest.mark.unit but reads a real fixture file; corrected to @pytest.mark.integration. Validation: ruff/format/pyright all green; pytest 67 passed (64 → 67, +3 regression tests). * bd_1-scaffolding-28p.9: US-009 — patterns & memory - .claude/rules/manifest-readers.md (new): Pydantic v2 frozen+extra=ignore for external-format readers; the three symlink-resolution traps caught by issue #2's pass-2 review (Path.relative_to vs resolve, RuntimeError on cycles, default paths must be canonicalised too); ManifestError + remediation: str pattern; no logging/metrics in stage-0 modules. - .claude/rules/testing-signal.md: appended sections on fixture regeneration via ephemeral uvx and the extra="forbid" drift detector pattern (DEC-005, DEC-009, DEC-012, DEC-017). - CLAUDE.md: marked issue #2 shipped; added "Public API surface (v0.1)" section listing signalforge.manifest as the first stable surface; cross-link to docs/manifest-loader-ops.md. Auto-memory entry added (outside the repo): the editable-install race that bit US-002's worker-in-worktree pattern — orchestrators must reinstall from the merged feature worktree before validating. * Address Copilot review comments on PR #15 - .claude/rules/manifest-readers.md:33: example used PathOutsideProjectError (doesn't exist); align with the real ModelPathOutsideProjectError so contributors copy/pasting the rule into a future external-format reader pick up the correct symbol. - src/signalforge/manifest/loader.py:35-41: module docstring claimed get_model caches a "unique_id -> Model" index. The actual implementation caches only path-based indexes (manifest.nodes is the unique_id lookup). Docstring updated to match. - plans/super/2-manifest-loader.md:7,521: replace absolute maintainer worktree paths (/home/wesd/...) with portable placeholders / generic prose so the doc isn't workstation-specific. ---------
* 3: BigQuery warehouse adapter (plan) Super plan for issue #3 — BigQuery warehouse adapter with sampling + dialect helpers. Covers Discovery + Architecture Review + Refinement Log + Detailed Breakdown. Phase 1 — DEC-001..012 lock the subpackage layout, ABC shape, sampling strategy, and ops surface. Phase 2 — six parallel reviews (security/performance/data-model/API/ observability/testing) surfaced 9 blockers + 18 concerns. No findings invalidate the Phase-1 shape. Phase 3 — DEC-013..028 resolve every blocker and concern. Notable: PartitionFilter ADT replaces raw SQL strings (B2); use_query_cache=False to preserve determinism (B3); DbtProfileTarget extra="forbid" + auth validator to make ADC fallback loud (B5); WarehouseAdapter.from_profile factory (B7); TestResult.explanation() to anchor explainable diffs (B8). Phase 4 — 14 stories, architecture-ordered. ~80 unit tests enumerated; 6 integration tests gated by SF_RUN_BQ. * 3: devolve plan to beads (epic + 14 tasks) Phase 5 → 7. Epic bd_1-scaffolding-8xk with 14 typed task children: US-001..US-012 implementation, US-013 Quality Gate, US-014 Patterns & Memory. Dependency graph wired so only US-001 is initially ready; predecessors unlock as they close. Plan doc Beads Manifest section now lists every bead ID and its deps. Phase: published → devolved. * bd_1-scaffolding-8xk.1: US-001 — Wire BigQuery deps + pytest bigquery marker Add google-cloud-bigquery and PyYAML runtime deps, types-PyYAML dev dep, register the `bigquery` pytest marker, and gate it from default collection via `-m 'not bigquery'`. Refs DEC-010, DEC-021 in plans/super/3-bigquery-adapter.md. * bd_1-scaffolding-8xk.2: US-002 — dbt profile YAML fixtures Hand-author six dbt profile fixtures under tests/fixtures/profiles/ for US-005's profile-loader tests: - bigquery_oauth.yml — minimal valid ADC profile (the v0.1 happy path). - bigquery_service_account.yml — non-oauth method; drives UnsupportedAuthMethodError (DEC-017). - multi_target.yml — two outputs; drives the target= override test. - missing_target.yml — target: dev with only prod defined; drives ProfileTargetNotFoundError. - dbt_project.yml — minimal project file for project-root resolution. - dbt_bigquery_drift_v1_9.yml — every documented dbt-bigquery 1.9 oauth field; drives the extra="forbid" strict-model drift detector. Mirrors the dbt-bigquery 1.9 docs and is bumped manually when dbt-bigquery releases a new minor — no regeneration script (DEC-009, DEC-017). Update tests/fixtures/README.md with a new "Profiles" section explaining the regeneration trigger, the source URL, and the hand-author rationale. All six files load cleanly via yaml.safe_load. Validation passes: ruff, ruff format, pyright (0/0/0), pytest (67 passed). * bd_1-scaffolding-8xk.3: US-003 — Warehouse errors module (DEC-026, DEC-022) Implements `signalforge.warehouse.errors`: a 16-class typed exception hierarchy rooted at `WarehouseError`, mirroring the manifest layer's remediation pattern. Every error carries a class-level `default_remediation` that the base `__str__` renders on a separate `↳ Remediation:` line — the explainable-diffs commitment, applied at the warehouse-adapter failure surface. DEC-022: user-supplied strings (table names, identifiers, profile fields, paths) route through a private `_format_value` helper that quotes via `repr()` so adversarial input cannot smuggle special characters into log viewers / stack traces. DEC-026: 15 typed subclasses listed in the plan are implemented; `ProfileTargetNotFoundError` inherits from `ProfileNotFoundError`, `SamplingRequiresPartitionFilterError` and `UnknownTableSizeError` inherit from `SamplingError`, so callers can catch with the parent when they don't care which it is. Also: - `signalforge.warehouse` package skeleton (`__init__.py` near-empty; US-011 will wire up the full re-export surface). - `pyproject.toml`: pytest `--import-mode=importlib` so `tests/manifest/test_errors.py` and `tests/warehouse/test_errors.py` can share a basename without adding `tests/__init__.py` (preserves the no-init rule from `testing-signal.md`). - 15 unit tests covering remediation rendering, repr-quoting of adversarial input, parent-catches-child semantics for the two inheritance chains, field accessibility on `BytesBilledExceededError`, and a `_CONSTRUCT_KWARGS` table that keeps every class in `__all__` exercised. Validation: ruff check + ruff format --check + pyright + pytest all pass (82 tests). * bd_1-scaffolding-8xk.4: US-004 — Warehouse models (DEC-003, 004, 013, 014, 016, 018, 020, 027) Implements the five typed return types for the warehouse adapter layer plus two private helper modules: - src/signalforge/warehouse/models.py — Dialect (+BIGQUERY_DIALECT constant), TableRef (+from_model gateway), PartitionFilter, ColumnStats, TestResult (with explanation()). - src/signalforge/warehouse/_sql_safety.py — DEC-013 identifier regex helper and run_test_sql validator. - src/signalforge/warehouse/_test_result_repr.py — DEC-020 type-aware compact row rendering used by TestResult.explanation. - tests/warehouse/test_models.py — 16 tests covering identifier validation, alias-over-name resolution, complex-type min/max, per-op PartitionFilter construction, and the explanation rendering surface. Mirrors signalforge.manifest.models conventions (Pydantic v2, frozen=True). TableRef/Dialect/PartitionFilter are frozen dataclasses (constructed by SignalForge code, not deserialised); ColumnStats/TestResult are Pydantic models so they can round-trip through the future JSON cache. TestResult.__test__ = False suppresses a pytest collection warning (class name starts with "Test" but it is a data class). Validation: ruff + ruff format + pyright + pytest all green; 98 tests pass. * bd_1-scaffolding-8xk.7: US-007 — FakeBigQueryClient (DEC-002, DEC-028) Hand-rolled fake for google.cloud.bigquery.Client with an explicit expect_query / expect_get_table / expect_list_rows API. Unexpected calls raise AssertionError; assert_all_expectations_met() flags unconsumed expectations. Lives in tests/warehouse/ — never imported by production code. Avoids two anti-patterns: pytest-bigquery-mock (dead, targets bq 2.x) and unittest.mock.MagicMock (auto-passes everything → always-pass tests, in violation of testing-signal.md). Adds six self-tests so regressions in the fake itself don't masquerade as adapter bugs upstream. Test count rises 98 → 104. * bd_1-scaffolding-8xk.5: US-005 — Profiles loader module (DEC-009, 017, 022, 023) Add `signalforge.warehouse.profiles` with `DbtProfileTarget` (Pydantic v2 frozen, `extra="forbid"`, `populate_by_name=True`, `method` field validator that raises `UnsupportedAuthMethodError` for anything other than `oauth`/None) and `load_profile(project_dir, target=None) -> DbtProfileTarget`. Resolution order per DEC-009: `$DBT_PROFILES_DIR/profiles.yml` (user-trusted) → `<project_dir>/profiles.yml` (symlink-hardened via `_path_safety`) → `~/.dbt/profiles.yml` (user-trusted). Active target = arg → profile's `target:` field → `ProfileTargetNotFoundError`. Profile name read from `<project_dir>/dbt_project.yml`'s `profile:` field. Soft 1 MB warning logged to `signalforge.warehouse` (DEC-023, threshold patchable from tests). `_path_safety.canonicalise_path` mirrors the manifest loader's `_canonicalise_path` precedent — three traps from `manifest-readers.md`: `.resolve()` before `.is_relative_to()`, wrap `RuntimeError` for symlink cycles, gate the default path the same as user-supplied input. Per DEC-017 the helper is copied (not imported) to keep subpackages decoupled; promotion to a shared utility is a US-014 follow-up. Tests: 13 new tests covering all three resolution paths, target override, missing target, missing-everywhere, unsupported method, unknown-field strictness, `schema:` alias, symlink rejection, soft warning, drift detector (test-only StrictModel mirroring every dbt-bigquery 1.9 oauth field), and logger discipline. Test count 98 → 111. `yaml.safe_load` only. * bd_1-scaffolding-8xk.6: US-006 — WarehouseAdapter ABC + factory Add `signalforge.warehouse.base.WarehouseAdapter` (abstract sampler / profiler / test-runner) plus the `from_profile` classmethod (DEC-019) that lazy-imports `BigQueryAdapter` for `profile.type == "bigquery"` and raises `UnsupportedProfileTypeError` otherwise. Concrete adapters land under `signalforge.warehouse.adapters/<warehouse>.py`; `bigquery.py` ships a skeleton — constructor + `__repr__` (DEC-022 credential redaction) + `dialect()` returning the live `BIGQUERY_DIALECT` constant — with the remaining abstract methods raising `NotImplementedError` until US-008. Tests cover ABC enforcement, factory dispatch, the unsupported-type branch, and both halves of DEC-019's max_bytes_billed fallback contract. Test count 117 → 122; ruff/format/pyright/pytest all green. * bd_1-scaffolding-8xk.8: US-008 — BigQueryAdapter full implementation Replace the US-006 stub with the full v0.1 BigQuery adapter: * `_client.py` — duck-typed `_BQClientProtocol` (matches both `bigquery.Client` and `FakeBigQueryClient`), `make_real_client`, `make_query_job_config` (DEC-015 defaults), `map_bq_exception` (`google.api_core` → typed `WarehouseError` translation), and `row_to_dict`. All `# pyright: ignore[...]` noise is contained here. * `bigquery.py` — full `BigQueryAdapter`: - `sample_rows` per DEC-006/DEC-024: deterministic `MOD(ABS(FARM_FINGERPRINT(TO_JSON_STRING(t))), bucket) < 1` with `Table.num_rows`-driven bucket sizing and fail-loud thresholds (`UnknownTableSizeError`, `SamplingRequiresPartitionFilterError`). - `column_stats` per DEC-008/DEC-025: requires active context, accumulates per-table, batched flush on first read; complex BQ types (`GEOGRAPHY`/`JSON`/`BYTES`/`ARRAY`/`STRUCT`/`RANGE`) skip `MIN/MAX` per DEC-016; soft `WARNING` at >500-column batches (DEC-023). - `run_test_sql` per DEC-007: wraps in `COUNT(*) [+ ARRAY_AGG]` aggregate; returns typed `TestResult` with `row_schema=None` (v0.2 will dry_run for inner schema). - DEC-013 identifier validation re-checked at `column_stats` entry; DEC-014 partition rendering by Python type; DEC-022-redacted `__repr__`; DEC-027 `project=None` resolves to client.project at quote time. * `tests/warehouse/test_bigquery_smoke.py` — 7 smoke tests pinning `_quote` resolution, `_render_partition_filter` rendering for date/datetime/str values (incl. `'`-escaping), the outside-context `RuntimeError`, and `__repr__` redaction. Comprehensive unit tests land in US-009. 129 tests pass (was 122). ruff/format/pyright all clean. * bd_1-scaffolding-8xk.11: US-011 — finalise signalforge.warehouse public API Replace the skeleton ``__all__: list[str] = []`` in ``signalforge/warehouse/__init__.py`` with a thin re-export layer mirroring ``signalforge.manifest`` (DEC-017): - 10 non-error names (``load_profile``, ``WarehouseAdapter``, ``BigQueryAdapter``, ``Dialect``, ``TableRef``, ``PartitionFilter``, ``ColumnStats``, ``TestResult``, ``DbtProfileTarget``, and the ``BIGQUERY_DIALECT`` constant) - the full 16-class ``WarehouseError`` hierarchy from ``signalforge.warehouse.errors`` ``__all__`` is a hard-coded sorted literal (not ``sorted([...])``) so pyright's ``reportUnsupportedDunderAll`` stays happy; sort order is guarded by a regression test instead. Add ``tests/warehouse/test_public_api.py`` mirroring ``tests/manifest/test_public_api.py``: six contract tests covering binding, no underscore leakage, sort order, helper isolation, the required-name minimum, and full re-export of ``signalforge.warehouse.errors.__all__``. Test count rises from 129 to 135. Validation: ruff check, ruff format --check, pyright, pytest all clean. * bd_1-scaffolding-8xk.10: US-010 — BigQuery integration tests (gated) Add six maintainer-only integration tests under `tests/warehouse/test_bigquery_integration.py` that exercise `BigQueryAdapter` against `bigquery-public-data.samples.shakespeare`. Each test wears both `@pytest.mark.bigquery` (filtered by the default `addopts = -m 'not bigquery'`) and `@pytest.mark.skipif(not SF_RUN_BQ)` (belt-and-suspenders per DEC-011/DEC-021): the default `pytest` run deselects all six and the test count stays at 129; `pytest -m bigquery` collects exactly the six new tests. Coverage: - `sample_rows` + `column_stats` round-trip on Shakespeare; - `run_test_sql` clean (`WHERE FALSE`) and dirty (`LIMIT 5`, `capture_failures=3`) paths; - `BytesBilledExceededError` via `max_bytes_billed=1` on Shakespeare (DEC-028 — free since BQ rejects the dry-run pre-flight); - `WarehouseAuthError` via `monkeypatch` on `google.auth.default` (DEC-028 — exercises the lazy-client construction path). The Shakespeare `TableRef` is built inside the test bodies via `__new__` + `object.__setattr__` to bypass DEC-013's strict identifier regex (real BQ project IDs may contain hyphens; the adapter's backtick-quoted `_quote` path handles them fine). Loosening the project-id regex is tracked separately. Add a "BigQuery integration tests" section to `CONTRIBUTING.md` documenting the `gcloud auth application-default login` + `SF_RUN_BQ=1 pytest -m bigquery` flow. * bd_1-scaffolding-8xk.12: US-012 — warehouse adapter ops doc Add docs/warehouse-adapter-ops.md per DEC-012 + DEC-027 covering quick start (ADC + load_profile + from_profile), profile resolution, cost defaults (max_bytes_billed, use_query_cache=False rationale, BQ job labels), sampling strategy with the TABLESAMPLE cost-asterisk and PartitionFilter use, the with-block contract for column_stats batching, SF_RUN_BQ-gated integration tests, debugging via the signalforge.warehouse logger and typed-error fields, and a full table of all 16 typed exceptions in signalforge.warehouse.errors. Cross-link from README's new Configuration section and from docs/research/dbt-research-index.md back to the ops doc and the design plan. Validation: ruff/format/pyright/pytest all green; test count unchanged at 129. * bd_1-scaffolding-8xk.9: US-009 — comprehensive BigQueryAdapter unit tests Adds tests/warehouse/conftest.py (centralised fake_client / adapter / table_ref / shakespeare_table fixtures) and tests/warehouse/test_bigquery_unit.py (45 tests covering cost defaults, _default_job_config DEC-015 plumbing, dialect identity, sample_rows DEC-006/DEC-024 decision tree, column_stats DEC-008/DEC-016/DEC-023 batching + complex-type MIN/MAX skipping, run_test_sql DEC-007/DEC-013 wrapping + SQL-safety rejects, exception mapping for BadRequest / NotFound / DefaultCredentials, and the DEC-025 context-manager lifecycle). Test count: 135 → 180 (+45). All four validation checks pass. Notes on plan deviations: * Skipped test_run_test_sql_populates_row_schema (v0.2 behaviour); replaced with test_run_test_sql_row_schema_is_none asserting the v0.1 contract. * Skipped test_per_call_max_bytes_caps_downward_only (v0.2). * test_default_max_bytes_billed_via_from_profile is already covered in test_base.py — not duplicated here. * SQL-text introspection uses a query-wrapper helper (_wrap_query_capture) rather than a custom regex matcher, since FakeBigQueryClient.expect_query re-compiles `matching` through re.compile and rejects non-Pattern objects. * test_column_stats_warns_at_threshold pre-seeds _column_stats_pending to trip the threshold; the simplified Option A flushes after every call so the pending list never grows past 1 via public API alone. * bd_1-scaffolding-8xk.13: Quality Gate — fix 8 findings from code review pass 1 1. (BLOCKER) TableRef now accepts hyphenated GCP project IDs via a new _PROJECT_RE / validate_project_id helper; strict identifier regex still gates dataset / table / column. Integration test drops the __new__ + object.__setattr__ bypass. 2. (MAJOR) _render_partition_filter escapes "\\" before "'" so an adversarial trailing "\\'" cannot terminate the BQ string literal early. 3. (MAJOR) Ops doc column_stats section now documents v0.1's eager-flush semantics (one query per call) and points at the v0.2 lazy-proxy follow-up. New "v0.2 follow-ups" section tracks both gaps. 4. (MAJOR) Ops doc cites the real signalforge_stage values (warehouse_sample, warehouse_stats, warehouse_test) and lists them under Cost defaults. 5. (MINOR) map_bq_exception accepts context={"max_bytes_billed": ...} so BytesBilledExceededError renders the configured cap instead of 0. 6. (MINOR) sample_rows adds ORDER BY FARM_FINGERPRINT(TO_JSON_STRING(t)) before LIMIT so truncation is deterministic when the bucket WHERE retains more than n rows (DEC-006). 7. (MINOR) NotFound "column" branch removed (dead in real BQ); missing columns now route through a BadRequest sub-branch matching "Unrecognized name" / "name not found". 8. (MINOR) make_query_job_config drops the # pragma: no cover; gains a default version (resolved from signalforge.__version__) and a unit test pinning use_query_cache=False, the maximum_bytes_billed value, and the labels dict. Validation: ruff check + ruff format --check + pyright + pytest all green; 186 default tests pass (was 180; +6 new tests across findings 1, 2, 5, 6, 7, 8) with the 6 BigQuery-marked tests still deselected. * bd_1-scaffolding-8xk.13: Quality Gate — fix 3 minor findings from review pass 2 - Drop legacy domain-scoped GCP project ID claim from _PROJECT_RE docstring; the regex never accepted them and _quote() doesn't split on ':'. Tracked as v0.2 follow-up. - Bound the strict-identifier fallback in _PROJECT_RE to 6-30 chars to match GCP's documented limits (was: any length). - Escape '\\' and "'" in compact_repr's string-value branch so TestResult.explanation() output is paste-safe per its docstring. Add regression test test_compact_repr_escapes_quotes_and_backslashes. - Update three tests using project="p" (1 char) to use "proj01" (6 chars) so they reach the dataset/name validation they target. * bd_1-scaffolding-8xk.13: untrack uv.lock (bark worktree artifact) uv.lock was inadvertently included in the prior Quality Gate commit; it's bark's worktree scaffolding output and isn't part of the SignalForge build (we use Hatchling + pip, not uv). Add to .gitignore. * bd_1-scaffolding-8xk.13: Quality Gate — fix 2 findings from review pass 3 - Centralise BQ string-literal escaping in _sql_safety.escape_bq_string_literal (handles backslash, single-quote, newline, CR, tab, NUL); use it from both _render_partition_filter and _test_result_repr. Newlines/control chars in partition values previously caused BigQuery syntax errors at execution time; the new helper renders all of them as escape sequences. Regression test test_render_partition_filter_escapes_newlines_and_tabs. - Wrap BigQueryAdapter.__exit__ in try/finally so a flush failure during clean exit cannot leave the adapter in a half-cleaned state — the next `with` block must start from empty caches per DEC-025. Regression test test_context_manager_exit_clears_caches_even_if_flush_raises. - Document Pass 3 finding 3 (drift detector self-consistency) as a v0.2 follow-up in docs/warehouse-adapter-ops.md alongside the legacy domain-scoped project ID gap. * bd_1-scaffolding-8xk.13: Quality Gate — fix phantom API from review pass 4 UnknownTableSizeError.default_remediation referenced an undefined adapter.refresh_table_metadata method, breaking Architectural Commitment #5 (explainable diffs — the remediation must actually work). - Implement BigQueryAdapter.refresh_table_metadata(table) — drops the cached Table for one ref so the next operation re-fetches num_rows. No-op outside an active context. - Add two regression tests: cache invalidation roundtrip + no-op outside context. - Drop the now-shipped GCP project-ID grammar entry from the v0.2 follow-ups list (it was completed in QG pass 1). * bd_1-scaffolding-8xk.14: Patterns & Memory — warehouse-adapters rule + CLAUDE.md surface Captures the conventions established by issue #3 into .claude/rules/warehouse-adapters.md: ABC + lazy-import factory, _client.py pyright containment, expect_* fake API, deterministic hash-mod sampling with fail-loud sizing, identifier validation at construction time, _default_job_config defaults (use_query_cache=False non-negotiable), typed error hierarchy with remediation + repr-quoted user input, and the explicit US-014 decision to keep _path_safety duplicated rather than extracted. Updates CLAUDE.md Repository status (two issues -> three) and Public API surface to include signalforge.warehouse re-exports. * PR #16: Address Copilot review comments Code: - TableRef: add `qualified_name` property (dialect-neutral `[project.]dataset.name`). Used by sampling errors so `.table` is a stable identifier rather than the dataclass repr. - bigquery.sample_rows: validate `n > 0` to prevent ZeroDivisionError on bucket sizing. - Map google.api_core exceptions with table identity from the call site: `map_bq_exception` now reads `context["table"]` so TableNotFoundError.table / ColumnNotFoundError.table carry the qualified name instead of the truncated message. Column name is extracted from BigQuery's "Unrecognized name: foo" via regex. - base.from_profile: replace `or 100_000_000` with explicit `is None` so an explicit `maximum_bytes_billed: 0` in the dbt profile is honoured. - profiles.load_profile: thread the resolved profiles.yml path into ProfileNotFoundError; ProfileTargetNotFoundError now lists the available output names in its remediation and exposes them via `.available` and `.profiles_path` fields. Docs: - warehouse-adapter-ops.md: correct the column_stats batching description (first call flushes every queued column for the table in a single query); narrow the BytesBilledExceededError doc to match what the mapping actually populates. - .claude/rules/warehouse-adapters.md: fix the FakeBigQueryClient snippet to match the real `expect_*` API; acknowledge the profiles.yml soft-size WARNING. Tests: - test_models: TableRef.qualified_name with and without project. - test_bigquery_unit: n<=0 ValueError; .table stable identifier on UnknownTableSize / SamplingRequiresPartitionFilter / TableNotFound; .column extraction on ColumnNotFound. - test_base: from_profile honours explicit maximum_bytes_billed=0. - test_profiles: ProfileTargetNotFoundError exposes .available, .profiles_path, and the remediation message contains every available target name. * PR #16: ruff format the new qualified_name test
* Add super plan for #4: PII safety layer Phase 1 locks library-only scope (no CLI, no LLM client — those land in #9 and #5). Phase 2 surfaced the load-bearing finding that schema-only mode leaks PII via column NAMES; Phase 3 resolves it with stable blake2b-hash placeholders. 26 decisions captured across config, fail- closed audit semantics, AuditEvent reproducibility (signalforge_version + policy_hash + audit_schema_version), extra=forbid on config-shaped models, and the SafetyPolicy.with_mode() seam #9 will use. Detailed Breakdown is 14 stories: scaffolding -> fixtures -> errors -> models -> SafetyPolicy -> config loader -> audit -> redact -> aggregate -> request builder -> public API + drift detector + AST scan -> docs -> quality gate -> patterns. TDD specified for every story with non-trivial business logic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update phase to published with PR #18 link Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Devolve plan to beads (epic + 14 tasks) Phase: devolved. Epic bd_1-scaffolding-o6f live; 14 tasks wired into the architecture-order DAG; bd ready confirms US-001 is the single entry point. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Populate Beads Manifest section; remove stale placeholder duplicates Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-0ix: Scaffold signalforge.safety subpackage + add safety pytest marker Empty __init__.py placeholder; full public re-exports land in US-011. New pytest marker enables `pytest -m safety` once tests exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-o64: Add safety-layer test fixtures (signalforge.yml variants, manifest with PII meta, audit JSONL sample) Hand-author the fixture corpora consumed by the upcoming PII-safety stories (US-005..US-011): eight signalforge.yml variants exercising the locked safety: top-level shape (DEC-025), the redact extend/replace mutual exclusion (DEC-017), the sampling-mode flag (DEC-021), and the DEC-013 path-traversal guard; a hand-derived manifest fixture carrying all four column-level PII opt-out signals plus a model-level signal (DEC-026); a one-line audit JSONL sample matching the locked AuditEvent shape (DEC-005 + DEC-014); and a deterministic regeneration script that will swap to the typed AuditEvent model once US-004 lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-rix: Add signalforge.safety.errors with 10-class hierarchy SafetyError base + 9 typed subclasses; mirrors WarehouseError/ManifestError patterns (default_remediation ClassVar, ↳ Remediation rendering, repr-quoted user input via _format_value helper). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-agc: Add signalforge.safety.models (SamplingMode, RedactionRecord, AuditEvent, LLMRequest) Frozen Pydantic v2 models with deep-immutable tuple sequences (DEC-022). AuditEvent carries reproducibility fields (signalforge_version, policy_hash, audit_schema_version=1) per DEC-014. LLMRequest docstring warns against direct construction (audit-completeness convention; AST scan in US-011). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-pfj: Add SafetyPolicy + _resolve_redact_patterns + _compute_policy_hash Frozen Pydantic v2 with extra=forbid; default mode=SCHEMA_ONLY; case-insensitive mode load; @model_validator resolves redact.extend/replace mutual exclusion; pattern-injection rejection (empty/*/?); with_mode() factory for #9's CLI; sample-mode WARNING; deterministic 16-hex policy hash for AuditEvent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-mg2: Add signalforge.safety.audit (fail-closed JSONL writer) O_APPEND atomic-append with fsync; mkdir parent at 0o700; file at 0o600. PIPE_BUF size cap (4000 bytes) raises AuditRecordTooLargeError. Any I/O exception propagates as AuditWriteError (DEC-011 fail-closed). Logger uses lazy-format with json.dumps to avoid ANSI/log-injection (DEC-022). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-ly6: Add signalforge.safety.config (load_safety_config + path safety) Full DEC-016 error contract: explicit-path miss raises ConfigNotFoundError; implicit miss / empty / missing-safety-key falls through to defaults; malformed YAML / non-mapping / schema-invalid raise typed errors. yaml.safe_load only. audit_path canonicalised + reject .. segments + require inside project_dir (DEC-013). _path_safety.py copied from warehouse layer per duplication precedent. * bd_1-scaffolding-y47: Add signalforge.safety.redact (classify + redact_rows + hash_column_name) _classify_column returns RedactionRecord | None; precedence column>model; case-insensitive tag matching; meta.contains_pii truthy coercion (DEBUG-log). Pattern match case-insensitive (lowercase both sides). hash_column_name uses blake2b-4 for stable 8-hex-char placeholders (DEC-010). redact_rows replaces values with '<REDACTED>' constant (no mutation). redact_column_names substitutes hashed names. Suspicious-unmatched-column WARNING heuristic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-3z5: Add aggregate_columns + FakeAdapter aggregate_columns wraps adapter.column_stats inside `with adapter:` (DEC-008 batching). Redacted columns return None keyed by hashed name; non-redacted keyed by real name. FakeAdapter mirrors warehouse FakeBigQueryClient's expect_* API; never MagicMock. ColumnNotInModelError on unknown column. Empty columns list short-circuits without ever opening the adapter context. When every requested column is redacted, the warehouse is never touched. The PII fixture's `database: "dev"` was bumped to `sf-demo-proj` so it satisfies BigQuery's project-ID grammar (6-30 chars, lowercase start) — the classify tests don't depend on the value, but TableRef.from_model now does. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-969: Add build_llm_request + default-mode regression suite Single entry point per DEC-009: classifies columns, dispatches per-mode (schema-only zero warehouse calls — DEC-012(c)), writes AuditEvent before returning (DEC-011 fail-closed). AuditEvent carries signalforge_version, policy_hash, audit_schema_version=1 (DEC-014). policy_flags populated from policy state (sample_mode_enabled, redaction_disabled, audit_path_overridden). Three default-mode regression tests cluster DEC-012 at policy/config/request layers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-2fb: Wire signalforge.safety public API + drift detector + AST scan __init__.py re-exports the documented surface (DEC-001). StrictAuditEvent drift detector validates the committed JSONL fixture and asserts field-set parity with production AuditEvent (DEC-026). AST scan rejects LLMRequest construction outside request.py — the audit-completeness convention from DEC-020(a). Includes negative test that the AST visitor catches a planted violation. Note: ``_path_safety`` is asserted absent from ``__all__`` rather than ``dir()``: Python attaches imported submodules to the parent package's namespace once any sibling (``config.py``) imports them, regardless of ``__init__.py``. Intent (private-helpers-stay-private) preserved via the ``__all__`` check, mirroring the warehouse package's pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-gx3: Add docs/safety-ops.md + README 'Data safety' section + .gitignore Operational reference for the PII safety layer matching manifest- and warehouse-ops doc precedent: default posture, modes, signalforge.yml schema, redaction patterns, four opt-out signals + precedence, column-name redaction (DEC-010 blake2b), audit JSONL schema with audit_schema_version=1, sensitivity caveat (column names are plaintext), rotation as user responsibility, debugging (logger levels), typed-error reference, CLI integration note pointing at #9. * bd_1-scaffolding-8av: Quality gate — fix bugs from code review Three blockers from quality-gate review: 1. LLMRequest.aggregates was dict[str, ColumnStats|None] — frozen=True only blocks attribute reassignment, not dict mutation. Downstream consumers (#5) could rewrite values after the audit log was written, desyncing the audit from what the LLM actually saw. Switched to tuple[tuple[str, ColumnStats|None], ...] for transitive immutability (DEC-022). Conversion happens at the request-builder boundary; aggregate_columns' public dict return is unchanged. 2. SafetyPolicy.with_mode() used model_copy(update=...) which silently skipped @model_validator(mode="after"). Calling policy.with_mode(SamplingMode.SAMPLE) — the documented CLI override seam (#9) — silently enabled sample mode without emitting the DEC-021 WARNING. Now goes through model_validate so the warning fires every time, regardless of construction path. 3. SafetyPolicy._normalise_mode returned non-string non-enum values unchanged, letting Pydantic raise a generic ValidationError instead of the typed InvalidSamplingModeError. Now every invalid type raises the typed error so the safety-layer hierarchy stays homogeneous. Plus: documented the .signalforge/ pre-existing-permissions caveat in docs/safety-ops.md (mkdir(exist_ok=True, mode=...) does not tighten pre-existing directories — user must verify pre-deploy). Twelve new regression tests: - test_safety_policy_with_mode_sample_emits_warning (DEC-021) - test_safety_policy_with_mode_schema_only_emits_no_warning - test_safety_policy_mode_non_string_non_enum_raises_invalid_sampling_mode_error (parametrised across 7 bad types: int, float, None, list, dict, tuple, object) - test_llm_request_aggregates_is_tuple_of_tuples_when_present (DEC-022) - test_llm_request_aggregates_immutable_when_none - test_llm_request_aggregates_field_reassignment_blocked_by_frozen Validation: 423 passed (up from 411), ruff/pyright/format all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-51d: US-014 — Patterns & Memory (rules/safety-layer.md + CLAUDE.md + bd remember) New .claude/rules/safety-layer.md distils the load-bearing patterns from issue #4: fail-closed audit semantics, column-name redaction via stable blake2b-4 hash, AuditEvent reproducibility fields (signalforge_version + policy_hash + audit_schema_version), extra=forbid on config-shaped models vs extra=ignore on read-back, the four opt-out signals + precedence, ANSI-safe lazy-format logger, AST audit-completeness scan, model_copy re-validation gotcha, signalforge.yml top-level namespace. CLAUDE.md updated: 'Repository status' bullet for issue #4, 'Public API surface (v0.1)' entries for signalforge.safety (SamplingMode, SafetyPolicy, LLMRequest, load_safety_config, build_llm_request, SafetyError hierarchy). Four bd remember entries: - schema-only mode redacts column names too - audit writes are fail-closed - Pydantic v2 extra=forbid (config) vs extra=ignore (read-back) split - model_copy doesn't re-run @model_validator(mode='after') Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address Copilot PR #18 review (3 real bugs + 2 doc fixes) Five Copilot review comments. Three real bugs and two doc inaccuracies: 1. config.py:101 (real bug) — load_safety_config's default-fallback branches returned SafetyPolicy() with the relative Path('.signalforge/audit.jsonl') field default, making the audit log resolve relative to CWD not project_dir. Now every default-fallback branch (no file / empty file / comments-only / missing safety: key / safety: present but no audit_path) canonicalises DEFAULT_AUDIT_PATH against project_dir before passing to SafetyPolicy. Symmetric with the user-override path. 2. config.py:136 (real bug) — audit_path: 123 (or [a,b], or true) crashed inside Path(audit_path_raw) with TypeError, leaking a non-SafetyError. Now type-checked at the gate: non-(str|os.PathLike) raises InvalidConfigError with a clear remediation pointing at YAML quoting. 3. config.py validator path (real bug) — Pydantic's extra="forbid" raises ValidationError with type='extra_forbidden', which the loader was wrapping as the generic PolicyValidationError. DEC-026 specified UnknownConfigKeyError for typos like `redacts:` or `mode_:`. Now the loader walks the Pydantic error list a second time and translates extra_forbidden into the typed UnknownConfigKeyError so the contract in safety-ops.md actually holds. 4. models.py:70 (doc bug) — RedactionRecord docstring claimed records are emitted for every column considered. They aren't — only redacted columns produce records (None pass-through for non-redacted). Updated docstring to match. 5. safety-ops.md:72 (doc bug) — aggregate-only example showed request.aggregates as a dict. After the Quality-Gate fix that made LLMRequest.aggregates a tuple-of-tuples (DEC-022 transitive immutability), the example was stale. Updated. The sixth Copilot comment (request.py:5) flagged stale PR metadata ('Phase: detailing (awaiting approval)' / 'Review the plan in this PR') that I had already fixed via REST API after the PR was opened. The title is now '4: PII safety layer' and the description reflects the shipped implementation. Will mark that comment as outdated. Eight new regression tests: - test_load_safety_config_no_file_default_audit_path_is_inside_project - test_load_safety_config_empty_file_default_audit_path_is_inside_project - test_load_safety_config_missing_safety_key_default_audit_path_is_inside_project - test_load_safety_config_no_audit_path_in_yaml_canonicalises_default - test_load_safety_config_typo_fixture_raises_unknown_config_key_error (renamed + tightened from generic SafetyError check) - test_load_safety_config_top_level_typo_raises_unknown_config_key_error - test_load_safety_config_audit_path_int_raises_invalid_config_error - test_load_safety_config_audit_path_list_raises_invalid_config_error - test_load_safety_config_audit_path_bool_raises_invalid_config_error Validation: 431 passed (up from 423), ruff/pyright/format all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add super plan for #5: LLM draft pipeline Plan doc covers Discovery, Architecture Review, 27 DECs across Refinement, and an 18-story Detailed Breakdown ready for devolve. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Devolve #5 plan to beads (epic + 18 tasks) Update plan-doc Phase marker to 'devolved' and fill in the Beads Manifest section with epic ID, the 18 task IDs (one per US-001..US-018), and the dependency graph. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-g11: US-001 — Subpackage scaffolding + anthropic dep + pytest markers Create empty signalforge.llm and signalforge.draft subpackages (placeholder __init__.py only; __all__ re-exports land in US-013). Add anthropic>=0.50,<1.0 to [project.dependencies]. Register pytest markers llm, draft, and anthropic; extend default addopts to also exclude the anthropic real-API smoke marker so default CI runs llm/draft but skips anthropic. Traces to: DEC-001, python-build.md, testing-signal.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-wsb: US-002 — Test fixtures (yml + manifest + golden response samples) * bd_1-scaffolding-4e2: US-003 — signalforge.llm.errors hierarchy * bd_1-scaffolding-zea: US-004 — signalforge.llm.models LLMResult Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-806: US-005 — signalforge.llm._client SDK shim (DEC-012 confinement) * bd_1-scaffolding-hhn: US-006 — signalforge.llm.client.call_anthropic centralized seam Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-wwx: US-007 — signalforge.draft.errors with bad-JSON envelope Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-nlz: US-008 — signalforge.draft.models CandidateSchema family + schema_version Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-buv: US-009 — signalforge.draft.config DraftConfig + load_draft_config Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-0kq: US-010 — signalforge.draft.prompts in-code template + version hash + envelope + mode-varying section Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-g27: US-011 — signalforge.draft.parser JSON validation + anchor-contract validator * bd_1-scaffolding-mtg: US-012 — signalforge.draft.audit LLMResponseEvent + fail-closed writer Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-9na: US-013 — signalforge.draft.schema integration + DraftOutcome + public API * bd_1-scaffolding-n98: US-014 — Audit-completeness AST scans + drift detector + cache-stability snapshot + grep gate * bd_1-scaffolding-eup: US-015 — Real-API smoke test (@pytest.mark.anthropic) Add tests/draft/test_smoke_real_api.py and the tests/fixtures/draft/smoke_manifest.json fixture it consumes. Gated by the ``anthropic`` marker (registered in pyproject.toml from US-001), excluded from default CI by the ``-m 'not bigquery and not anthropic'`` filter, runnable locally with ``pytest -m anthropic`` when ``ANTHROPIC_API_KEY`` is set. Path A (wire test): the cached manifest summary for the synthetic ``simple_orders`` model is ~100 tokens, far below Haiku's 2048-token minimum and Sonnet's 1024-token minimum. The pre-send ``count_tokens`` call hits the real Anthropic API (proving SDK + auth + transport), then ``call_anthropic`` raises ``LLMCacheTooSmallError`` before any ``messages.create`` is issued. This is documented in the test docstring as a deliberate v0.1 constraint; when the smoke fixture grows above 2048 tokens the test can be upgraded to Path C (full round-trip) without changing its marker or invocation. Fixture exercises the no-neighbours code path of ``_render_manifest_summary`` (empty ``depends_on.nodes``, empty ``refs``). Validation: ruff/ruff-format/pyright clean. ``pytest`` default still reports 608 passed; ``deselected`` count goes from 6 to 7 to reflect the new ``anthropic``-marked test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bhu: US-016 — docs/draft-ops.md + README + CLAUDE.md * bd_1-scaffolding-273: US-017 — Quality gate fixes from code review pass 1 Three real bugs surfaced and fixed (pass 2 truncated by org usage limit; all four validation checks green). 1. parser._validate_anchor_contract: a CandidateColumn whose `name` was not in `model_columns` slipped through (the membership check lived only inside an `elif` after the parent-column-mismatch branch). Now the column name itself is validated, and the nonexistent-column check on each test is independent of the parent-column-mismatch check so a hallucinated column surfaces both violations. 2. client.call_anthropic: the cache-anomaly WARNING fired on every cache HIT. `cache_creation == 0` and `cache_read > 0` is the normal healthy case (the cache was created on a prior call). The warning now requires both creation AND read to be zero — the genuine no-op signal. 3. prompts._render_dynamic_block: a Model.raw_code containing the literal `</MODEL_SQL>` would terminate the prompt-injection envelope early and let downstream content escape the data fence. New typed error PromptEnvelopeBreachError; render is now refused before any LLM call. Tests: 608 → 611 (one new test per fix). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-x3t: US-018 — Patterns & Memory (.claude/rules/llm-drafter.md + bd remember) Distil load-bearing patterns from issue #5 into a project rule that future contributors (human + Claude) consult before touching signalforge.llm or signalforge.draft. Mirrors safety-layer.md shape. Sections: SDK seam confinement (DEC-012) | retry test-determinism via module aliases (DEC-004) | fail-closed response audit (DEC-006/008/013) with the propagation-IS-the-defence rule | <MODEL_SQL> envelope + breach guard (DEC-007) | cached-block scope + 8000-token cap (DEC-009) | cache-anomaly dual-zero rule (post-QG fix to DEC-014) | whole-draft fail-loud anchor contract with collected violations (DEC-003/022) | ANSI-safe lazy-format logger gate (DEC-011) | four AST audit-completeness scans (DEC-013) | signalforge.yml top-level llm: namespace (DEC-027). Plus 5 persistent bd memories (issue-5-llm-*) so cross-conversation recall is anchored on the load-bearing patterns, not the file paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-273: PR review fixes (Copilot + CodeRabbit, 12 threads) All 12 review threads addressed; none deferred. Two require behaviour notes; the rest are mechanical hardening of existing seams + tests. PROMPT_VERSION rotated 4800590d3d749955 -> 1c55806467984090 because the system prompt's claim that "tests without rationale are rejected by the parser" was softened (the parser does not, in fact, enforce it — DEC-026 documented this as a soft constraint owned by the grader at #7). Cache-stability test pinned to the new value. Real bug fixes: - src/signalforge/draft/schema.py: sent_sql_hash now hashes Model.raw_code (not the dynamic block which carries envelope + data section). Drift would have broken correlation with raw_code. - src/signalforge/draft/models.py: schema_version: Literal[1] = 1 so payloads with schema_version != 1 fail validation rather than silently passing. - src/signalforge/llm/client.py: count_tokens now MAPS Anthropic exceptions to typed LLMError subclasses (no retry on the probe call — consuming the create-budget on a probe failure would let one blip exhaust the budget). Plus per-class retry counters (attempt_429 / attempt_5xx / attempt_conn) so one failure class cannot consume another's budget; total_attempts drives the backoff math + WARNING log so delays remain monotonic across mixed types. - src/signalforge/draft/prompts.py: removed the false claim that the parser rejects rationale-less tests. New phrasing matches actual enforcement (parser is permissive; grader scores). Hardened test infrastructure: - tests/llm/test_logger_grep_gate.py + tests/llm/test_client.py: regex now matches every f-string prefix permutation (f, F, rf, fr, rF, FR, etc.) AND optional whitespace after the opening paren AND single or double quotes. Previous regex only caught (f". Bypassable by switching quote style. - tests/test_audit_completeness.py: AST scan handles import aliasing (import anthropic as a; a.Anthropic(...)) AND direct-symbol imports (from anthropic import Anthropic; Anthropic(...)). Previous matcher could be silently bypassed. - tests/test_audit_completeness.py: exclusion lists now path-relative (request.py / _client.py / audit.py at root of scan dir) rather than basename-only — prevents accidental shadowing of nested files with the same basename. Trivial fixes: - README.md: MD040 fence language (text). - tests/fixtures/draft/regenerate.sh: comment now reflects current reality (US-015 smoke test exists; refresh is still manual until fixture grows past cache minimums). New tests: - test_call_anthropic_per_class_budgets_do_not_cross_consume: pins the invariant that one class's failures cannot consume another's budget. 612 passed (was 611 + 1 new). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 6: super-plan for prune engine Phase-4 plan for issue #6 — drop always-pass and known-clean-fail candidate tests against warehouse data. Covers the load-bearing "signal over volume" commitment from CLAUDE.md. Architecture review surfaced two pivots resolved in DEC-012/DEC-013: the deterministic-sample predicate (TO_JSON_STRING(t)) reads every column, so US-003 live-verifies the cost model against bigquery- public-data before locking strategy; the warehouse adapter's QueryJobConfig.job_timeout_ms plumbing folds in as US-002 (~10 LOC) so the per-test budget actually enforces. 28 decisions, 16 stories (14 implementation + Quality Gate + Patterns & Memory). Stories trace to DEC-### and embed TDD on every logic-shaped surface (compiler, engine, audit, errors, models, config). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6: bump plan phase to published PR #20 is open; phase marker updated for re-invocation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6: devolve plan — beads manifest Epic bd_1-scaffolding-y8y + 14 implementation tasks + QG + PM. Dependencies wired so `bd ready` returns only US-001 at start. Phase marker bumped to devolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.1: scaffold signalforge.prune subpackage Empty docstring-only stubs for engine, compiler, models, errors, audit, and config modules. Subsequent stories (US-002 ... US-014) land the actual logic. Plan: plans/super/6-prune-engine.md (DEC-001). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.4: PruneResult / PruneDecision / primitives Frozen Pydantic v2 models for the prune layer's read-back surface. DropReason and Scope discriminator literals. Computed-property aggregates (kept_decisions, dropped_decisions, kept_count, etc.) for the diff renderer (#8). PruneDecision carries the typed CandidateTest discriminated union (DEC-004), not a loose dict. Plan: plans/super/6-prune-engine.md (DEC-003, DEC-004, DEC-014, DEC-015). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.6: PruneError hierarchy Six typed exception classes: PruneError, PruneConfigError, PruneTrustedModelNotFoundError, PruneTimeoutError, PruneAuditWriteError, PruneAuditRecordTooLargeError. Each carries default_remediation; __str__ renders message + remediation; user input is repr()-quoted to defeat ANSI injection. Introduces signalforge.errors.SignalForgeError as the project-wide root so PruneError(SignalForgeError) wires per DEC-006. Existing layer roots (SafetyError, DraftError, WarehouseError, ...) stay untouched per task scope; future stories can rebase them. Plan: plans/super/6-prune-engine.md (DEC-006, DEC-022). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.6: drop spurious signalforge.errors module The worker's first commit added signalforge/errors.SignalForgeError to satisfy the plan's PruneError(SignalForgeError) note, but every existing layer (Safety/Draft/Warehouse/Manifest) subclasses Exception directly with no shared root. Adding a project-wide root for prune alone left the codebase inconsistent. Match the established pattern: PruneError(Exception). The plan note referencing SignalForgeError was a planning bug; the precedent in the other layers is the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.7: compiler — failing-rows SQL x 4 test types Pure SQL transform: not_null, unique, accepted_values, relationships to BigQuery failing-rows SELECTs. Dialect-driven (Dialect.quote_char) so v0.2 adapters drop in without changes. Reuses warehouse._sql_safety.escape_bq_string_literal for accepted_values literals (DEC-024). _RequiresFutureData sentinel for relationships(to: unknown) instead of an exception (DEC-006). NULL- exclusion matches dbt-core verbatim (DEC-023). Plan: plans/super/6-prune-engine.md (DEC-023, DEC-024, DEC-025, DEC-026). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.8: prune audit — fail-closed prune.jsonl writer PruneEvent + _write_prune_event(event, path) with O_APPEND|O_CREAT| 0o600, fsync, size cap (4000 bytes) before os.open, no try/except. Mirrors safety + draft fail-closed audit semantics. _build_prune_event is the single construction seam (DEC-018). _compute_config_hash uses sha256(...)[:16] to match safety.policy_hash (DEC-005). AST audit-completeness scan extended (tests/test_audit_completeness.py) with a fifth scan: PruneEvent construction confined to signalforge/prune/audit.py only. Plan: plans/super/6-prune-engine.md (DEC-007, DEC-014, DEC-016, DEC-018). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.5: PruneConfig + load_prune_config User-facing config for the prune layer. Inner PruneConfig is extra="forbid" so typos fail loud; outer _PruneConfigFile wrapper is extra="ignore" so sibling stages coexist (DEC-015, DEC-020). Defaults match Phase-1 housekeeping: scope=sample, sample_size=100k, test_timeout=30s, total_budget=600s, capture_failure_rows=3, trusted_models=(), partition_filter=None. yaml.safe_load only. Trusted-models validation against the manifest is NOT done at load time — that's prune_tests() entry (DEC-008, US-009). Plan: plans/super/6-prune-engine.md (DEC-009, DEC-015, DEC-020). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.5: drop dead _DEFAULT_CONFIG_FILENAME constant Worker left a stale module-level constant from comparing with draft/config.py (which uses project_dir + filename resolution). Our load_prune_config takes the path directly, so the constant was never read. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.2: warehouse adapter timeout_ms plumbing Extend make_query_job_config and _default_job_config with a timeout_ms: int | None = None kwarg threaded into QueryJobConfig.job_timeout_ms. Default None (existing behaviour unchanged). The prune layer (#6 US-009) will use this for per-test budget enforcement (Q5=A, DEC-013). BigQuery cancels server-side at expiry; bytes-billed through cancellation are NOT refunded. _default_job_config is now keyword-only on (stage, timeout_ms); internal call sites in bigquery.py and the existing positional calls in tests/warehouse/test_bigquery_unit.py updated to match. Pyright noise for the loosely-typed job_timeout_ms attribute stays confined to _client.py per the SDK-seam convention. Plan: plans/super/6-prune-engine.md (DEC-013, AR-B2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.9: prune engine — prune_tests orchestrator End-to-end integration of compiler + adapter + audit + budget into prune_tests(model, adapter, candidates, manifest, *, config, audit_path) -> PruneResult. Routes outcomes through the five DropReason values per the plan's decision matrix: * 0 failures → drop (always-passes) * relationships(to: ?) → drop (requires-future-data) [no WH call] * fail + trusted → drop (failed-on-known-clean-data) * fail + untrusted → keep (kept) * WarehouseError → keep (kept-without-evidence) * total-budget exceed → keep (kept-without-evidence) [no WH call] trusted_models validation at entry (DEC-008) — typo'd unique_id raises PruneTrustedModelNotFoundError before any warehouse call. Audit fail-closed (DEC-016): OSError → PruneAuditWriteError(cause); PruneAuditRecordTooLargeError propagates raw. Either aborts the run. Module-level _sleep / _now_monotonic_ms aliases (DEC-019) for deterministic test override. Per-test timeout_ms threading from config.test_timeout_seconds into adapter.run_test_sql is deferred — adapter doesn't expose it yet (US-002 plumbed make_query_job_config; surfacing through run_test_sql is v0.2). For v0.1, per-test enforcement is implicit via WarehouseError catch; the total-budget gate handles wall-clock. Plan: plans/super/6-prune-engine.md (DEC-002, DEC-008, DEC-011, DEC-016, DEC-019). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.13: signalforge.prune public re-exports Re-export the v0.1 public surface from signalforge.prune: * prune_tests (orchestrator entry) * PruneResult / PruneDecision / DropReason / Scope (read-back models) * PruneConfig / load_prune_config (user config) * PruneEvent (audit type; constructed only in audit.py per AST gate) * PruneError + five concrete subclasses Internal helpers (_compile_test, _write_prune_event, _sleep, etc.) stay reachable via dotted import for tests but are absent from the package namespace -- DEC-021. Smoke test (tests/prune/test_smoke.py) pins the surface against __all__ and ensures internal helpers don't leak. Plan: plans/super/6-prune-engine.md (DEC-021). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.11: extend logger grep gate to signalforge.prune Add src/signalforge/prune/ to the lazy-format-logger scan. Renames the test to test_no_f_string_logger_calls_in_llm_draft_or_prune_modules to reflect coverage. Mirrors the existing DEC-022 / DEC-011 gate across all three subpackages that emit observability. Plan: plans/super/6-prune-engine.md (DEC-017). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.10: drift detectors for prune read-back models Three extra="forbid" Strict mirrors (StrictPruneResult, StrictPruneDecision, StrictPruneEvent) paired with committed JSON/ JSONL fixtures covering all five DropReason values. Field-set parity tests catch silent schema drift before a v0.2 reader does. Mirrors tests/safety/test_drift_detector.py shape exactly. PruneConfig is already extra="forbid" in production (DEC-015), so no drift gate is needed there. Plan: plans/super/6-prune-engine.md (DEC-010, DEC-015). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.3: diagnostic cost probe + ops-doc scaffold Add tests/warehouse/test_sample_cost_probe.py — measures total_bytes_billed for a deterministic 100k-row sample against bigquery-public-data.iowa_liquor_sales.sales. Marked @pytest.mark.bigquery + skipif(SF_RUN_BQ) so default CI excludes it. Used to verify or refute Phase-1's "sample-mode is cheap" assumption (AR-B1) — TO_JSON_STRING(t) reads the whole row. Pre-creates docs/prune-ops.md with the standard ops-doc skeleton: public-API placeholder, drop-reason taxonomy, cost-model section (awaits the probe's figure), v0.2 deferrals. US-014 fills the body. Plan: plans/super/6-prune-engine.md (DEC-012, AR-B1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.12: BQ integration test against bigquery-public-data Single end-to-end test runs prune_tests against the public Iowa liquor sales dataset with two candidate tests: * not_null(invoice_and_item_number) -> always-passes drop * accepted_values(category_name, values=("VODKA", "GIN")) -> kept Confirms the orchestrator's decision matrix in real warehouse conditions. Marker @pytest.mark.bigquery + skipif(SF_RUN_BQ) keeps it out of default CI; runs only when SF_RUN_BQ=1 and ADC are configured (mirrors tests/warehouse/test_bigquery_integration.py). Plan: plans/super/6-prune-engine.md (acceptance criterion #7, AR-C11). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.14: docs/prune-ops.md + CLAUDE.md update Fill out the operational reference for the prune layer. Cover the public API surface, signalforge.yml prune: config block, drop-reason taxonomy, audit JSONL schema, cost-model verification path, real-warehouse test invocation, and v0.2 deferrals. Update CLAUDE.md's Repository status and Public API surface (v0.1) sections to list the prune layer alongside the manifest, warehouse, safety, and draft layers. Plan: plans/super/6-prune-engine.md (DEC-027, all v0.1 surfaces). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6/QG: fix bugs surfaced by 4-pass code review Seven fixes across the prune layer's surface and tests: * Custom PruneResult.__repr__ redacts compiled_sql and sample_failures so accidental log-line interpolation can't leak SQL or sampled rows (DEC-022). * Audit path canonicalised via warehouse._path_safety before write so a symlinked .signalforge/prune.jsonl can't redirect writes outside the project (DEC-016). * load_prune_config(project_dir, path=None) signature aligned with load_safety_config / load_draft_config — one calling convention across stages. * Default audit_path resolves relative to project_dir (defaulted to cwd) instead of cwd directly — matches safety + draft fix. * Compiler identifier shape validation: CandidateTest column/field/to routed through warehouse._sql_safety.validate_identifier; an adversarial identifier returns the new _InvalidIdentifier sentinel which the engine routes to kept-without-evidence (DEC-024). * test_yaml_safe_load_rejects_python_objects strengthened to use a side-effect-detectable gadget (Path.touch on a tmp marker); test now actually fails if yaml.load is substituted for yaml.safe_load. * PruneTimeoutError docstring documents it as forward-compat for v0.2 (kept on public surface; no v0.1 callers see it). Plan: plans/super/6-prune-engine.md (Quality Gate of issue #6). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-y8y.16: distill prune-layer rules into .claude/rules New .claude/rules/prune-engine.md captures the load-bearing conventions established by issue #6: * kept-without-evidence routes to decision="kept" (DEC-011) * fail-closed prune.jsonl audit (DEC-016) * symlink-hardened audit path via canonicalise_path (post-QG) * identifier shape validation at compile seam (post-QG) * dialect-driven compiler (no BigQuery-isms) * single AST scan per new audit-event type (DEC-018) * ANSI-safe lazy-format logger + grep gate (DEC-017) * custom __repr__ on result-shaped models (post-QG, DEC-022) * drift detectors mandatory for extra="ignore" models (DEC-010) * API alignment with adjacent stages (load_*_config signature, project-dir-relative default paths) * signalforge.yml prune: namespace (DEC-020) CLAUDE.md cross-references the new rule file. Plan: plans/super/6-prune-engine.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6/QG: wire prune scope/sample_size/partition_filter into compiled SQL PR #20 review (Copilot + CodeRabbit) flagged that PruneConfig.scope was advisory-only — the engine wrote decision.scope into audit metadata but never wrapped the failing-rows SQL with the deterministic-sample CTE. Sample-mode tests ran against the FULL table regardless of config, defeating the cost model US-003 was built on. The compiler is now the single seam for the wrapping. _compile_test accepts scope, sample_size, sample_bucket, and partition_filter; sample mode wraps the test in a `WITH sample AS (SELECT * FROM <table> AS t WHERE MOD(ABS(FARM_FINGERPRINT(TO_JSON_STRING(t))), <bucket>) < 1 [AND <partition>] LIMIT <size>) <test>` compound matching the warehouse adapter's sample_rows shape (DEC-006 of issue #3). Full mode + a partition_filter composes via a derived table so per-test WHERE-clause shapes don't have to be edited. The orchestrator computes sample_bucket once per run from Table.num_rows / sample_size via the SDK shim seam (acknowledged minor encapsulation crack, documented inline; v0.2 may add a public WarehouseAdapter.get_table_metadata seam). When num_rows is unavailable in sample mode the orchestrator raises PruneError rather than silently degrade to "every row" — that fail-loud signal lands in front of the operator. Sample-mode relationships samples the CHILD only (the parent stays at full so an orphan detected in the child sample is not a false positive caused by the parent's missing-from-sample row). Also fixes the relationships parent-resolution ambiguity case: when two or more manifest models share Model.name (cross-package collision), the compiler returned the FIRST match silently. It now returns _RequiresFutureData with a precise count, routing the test to "requires-future-data" rather than picking a wrong parent. Pinned snapshot fixtures for each of the four candidate-test variants in sample mode (not_null_sample.sql, unique_sample.sql, accepted_values_sample.sql, relationships_sample.sql). 15 new tests across compiler + engine; existing tests (which were implicit-default sample-mode but tested full-mode SQL shapes) updated to scope="full" so they continue to test the legacy unwrapped path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6/QG: prune audit short-write loop + SF_RUN_BQ truthy guard + BQ probe race fix Three independent PR-#20 review items grouped because they all affect the I/O / opt-in seams: (1) signalforge.prune.audit._write_prune_event now loops on os.write until the full payload lands. POSIX write(2) may return fewer bytes than requested (EINTR on signal-interrupted calls; short writes on some filesystems / kernels); the previous single-call assumption could silently truncate the JSONL line. A persistent zero-byte return raises OSError so we don't spin forever. (2) The SF_RUN_BQ env-var check in three integration test files now restricts truthy values to {"1", "true", "yes", "on"} (case-insensitive) via a _bq_runs_enabled() helper. The previous `not os.environ.get(...)` treated SF_RUN_BQ=0 / SF_RUN_BQ=false / SF_RUN_BQ=no as truthy because they're non-empty strings — surprising for a user trying to disable the runs. (3) tests/warehouse/test_sample_cost_probe.py now reads total_bytes_billed directly off the QueryJob instance returned by client.query(...) rather than via INFORMATION_SCHEMA.JOBS_BY_USER. The previous lookup filtered by signalforge_stage='warehouse_sample' label and a creation-time lower-bound, which could attribute bytes to the wrong job (a leftover prune-stage run, or a concurrent process emitting the same label). Reading total_bytes_billed straight off the just-issued QueryJob eliminates the race entirely. The probe deliberately bypasses the adapter's public sample_rows path to access job stats; documented inline as a v0.2 deferral (DEC-027 of issue #6). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 6/QG: docs/rules corrections + rename make_query_job_config to _make_query_job_config PR #20 review (Copilot + CodeRabbit) flagged six accuracy items in the documentation, rules, and adapter docstrings: (.claude/rules/prune-engine.md) - requires-future-data was described as "Kept" — it's actually "Dropped". - engine.py::_classify_decision was a fictional reference; renamed to the actual helper (_decide_from_test_result). - The signalforge.yml prune-block field list referenced phantom keys (audit_path, mode); replaced with the real PruneConfig fields. (src/signalforge/prune/config.py) - The trusted_models docstring contradicted the implementation. Trusted models route to "failed-on-known-clean-data" (drop, presumed buggy test); the docstring incorrectly claimed they "surface as real failures rather than the drop". Rewritten to match. (docs/prune-ops.md) - prune_tests signature was missing the project_dir parameter. - The test_timeout_seconds entry was unclear about what's wired vs deferred to v0.2; clarified that v0.1 has no per-test timeout enforcement (run_test_sql does not yet accept timeout_ms; the knob is reserved for v0.2 alongside the existing _make_query_job_config plumbing). (docs/warehouse-adapter-ops.md) - The _default_job_config example was missing the required stage= kwarg. - Removed the false claim that "the prune layer (#6) uses this for per-test budget enforcement" — issue #6 ships with total_budget_seconds enforcement only. (src/signalforge/warehouse/adapters/bigquery.py) - _default_job_config docstring claimed prune layer was the only caller supplying a non-None timeout_ms; rewritten to match reality (currently used only by tests; reserved for v0.2 prune integration). (rename make_query_job_config → _make_query_job_config) - The function lives inside the SDK seam (_client.py) and is internal to the adapter layer. The privacy-prefix convention dictates the underscore. Mechanical rename across _client.py, bigquery.py, prune/engine.py docstring, and the two test files that import it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 21: Run AR-B1 cost-model probe + fix BQ adapter TableRef bug Records the verified bytes-billed figure for US-003 in docs/prune-ops.md and bundles the adapter bug surfaced by the live run. - AR-B1 probe ran 2026-05-01 against bigquery-public-data.iowa_liquor_sales.sales (~30M rows, ~24 cols) for a 100k-row deterministic sample. BigQuery's pre-execution analyzer estimated 9,924,771,840 bytes (~9.92 GB) — ~99x the Phase-1 estimate and ~2x the probe's 5 GB sanity ceiling. The adapter's 100 MB cost cap (DEC-005 of #3) blocked execution before any bytes were billed; the figure is captured from the bytesBilledLimitExceeded error. - docs/prune-ops.md Cost model section: replaced the TBD placeholder with the verified figure and run date, and rewrote the surrounding paragraphs to call out that Q4=A is NOT adequate for v0.1 sample-mode on wide tables. Schema-only stays the v0.1 default. - Adapter bug (separate from US-003): BigQueryAdapter._get_table and the probe were passing our Pydantic TableRef directly into google.cloud.bigquery.Client.get_table, which only accepts str | TableReference | Table | TableListItem and explodes on TableRef.path access. Both call sites now pass qualified_name. FakeBigQueryClient._coerce_to_tableref learned to parse the dotted string, so the unit suite still binds the production path. The same shape was present in the integration tests at tests/warehouse/test_bigquery_integration.py — those would have failed identically against live BQ; they're now reachable as a side-effect. - Q4=C escalation tracked in #22 (temp-table-materialised sample for v0.2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 21: Address PR review feedback - _coerce_to_tableref accepts 2-part 'dataset.table' refs (Pydantic TableRef.qualified_name omits the project segment when project=None); raises a clear AssertionError for other shapes. (Copilot, CodeRabbit) - docs/prune-ops.md: lead with the BigQuery reason code bytesBilledLimitExceeded as the durable identifier; flag that the SDK exception class is unstable across versions (adapter expects BadRequest/400, live 2026-05-01 run with google-cloud-bigquery 3.41.0 raised InternalServerError/500). (Copilot, CodeRabbit) - docs/prune-ops.md: replace the wrong cost_limit_bytes reference with the actual profile-level maximum_bytes_billed field; point at warehouse-adapter-ops for context. (CodeRabbit) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 21: Add warehouse-adapters rule on TableRef vs vendor SDKs Captures the lesson from the AR-B1 cost-probe failure: don't pass our Pydantic TableRef directly into vendor SDK methods that accept str | TableReference | Table | TableListItem. Always pass ref.qualified_name. Integration tests against the real SDK are the only thing that catches this; FakeBigQueryClient won't. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 7: Add super plan for quality grader 29 decisions captured (DEC-001 .. DEC-029) across discovery, architecture review, and refinement. 13 stories generated covering scaffold, typed models, rubric, config, prompts + envelope guard, parser, fail-closed audit, orchestrator + sidecar JSON, AST/logger gate extensions, real-API smoke, ops doc, quality gate, and patterns & memory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 7: Update plan phase to published with PR #24 link Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 7: Devolve plan to beads (epic + 13 tasks) Phase: devolved. US-001 (scaffold) is the initial ready entry point; US-013 (Patterns & Memory) runs last after Quality Gate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.1: Scaffold signalforge.grade + 9-class GradeError hierarchy Mirror prune/errors.py shape: classvar default_remediation, repr-quoted user-supplied strings via _format_value(), __str__ renders message + ↳ Remediation: line. Module docstring on __init__.py documents the safety redaction boundary closed at draft (DEC-013). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.2: Typed grade models + drift detectors + fixtures GradingResult / GradingReport / GradeEvent (Pydantic v2, extra=ignore, frozen). Score is float | None to support DEC-015 degraded path. Computed fields one_line_why, pass_rate, mean_score, aggregate_complete, passed (mirrors clauditor aggregate-AND threshold). Minimal __repr__ on every result-shaped model (DEC-022 of #6). Strict<X>(extra=forbid) drift detectors validated against committed fixtures (grade_event_v1.jsonl, grade_report_v1.json). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.3: Rubric data model + DEFAULT_RUBRIC + canonical hash Criterion (frozen, extra=forbid, both id and criterion required and non-empty per DEC-017). GradeThresholds with [0.0, 1.0]-bounded min_pass_rate=0.7 / min_mean_score=0.5. Rubric: TypeAlias = tuple[ Criterion, ...] per DEC-011 (no wrapper class). DEFAULT_RUBRIC carries the four locked criteria from DEC-016 verbatim (clarity, consistency, rationale, no-redundant); criterion text pinned by a verbatim-match test for reproducibility. _canonical_rubric_hash sorts by id then JSON-dumps with sort_keys for deterministic order-invariant blake2b-8 hex (DEC-010). Pinned to a golden hex regression test so changes to DEC-016 text break the build. validate_rubric rejects duplicate ids with GradeRubricError. The degenerate empty rubric also raises GradeRubricError: an empty rubric would make every grade run a silent no-op, which is exactly the fail-loud-on-typo posture the rest of the layer takes (extra=forbid on config-shaped models, etc.). Recording the choice here so a future "empty rubric is fine, just a no-op" patch surfaces this rationale. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.5: Prompt rendering + <ARTIFACT> envelope + version hashes System prompt + cached rubric block + dynamic per-(artifact, criterion) block. Dynamic block fences artifact_text inside <ARTIFACT>...</ARTIFACT>. GradePromptEnvelopeBreachError raised BEFORE rendering if literal </ARTIFACT> appears in artifact_text — the only LLM-prompt defence between LLM-generated artifact content and the judge prompt (DEC-008). prompt_version_template (per-run constant per rubric) and criterion_prompt_hash (per-criterion stable across artifacts) per DEC-019, both pinned to golden hex strings as regression detectors. extract_artifact_text resolves DEC-009 dotted-path artifact_ids against a post-prune CandidateSchema; unknown ids raise GradeOutputError with violation_type unknown_artifact_id. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.6: Response parser + single-criterion anchor contract parse_grade_response: strip optional JSON code fence, json.loads, validate shape (dict with required criterion_id/score/passed; optional evidence/reasoning default empty), enforce single-key anchor returned.criterion_id == sent.criterion_id, validate score range [0.0, 1.0] excluding nan/inf and bool-disguised-as-number, validate passed is strict bool. Returns typed GradingResult. GradeOutputError.violation_type tightened to Literal taxonomy of nine values (json_parse, missing_required_field, missing_criterion_id, criterion_id_mismatch, score_out_of_range, score_not_a_number, passed_not_a_bool, unknown_artifact_id, ambiguous_artifact_id). The last two preserved from US-005's extract_artifact_text call sites. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.7: Fail-closed grade audit writer + single GradeEvent seam write_grade_event mirrors prune.audit.write_prune_event verbatim: size-cap before os.open (4000 bytes; oversize leaves NO artifact), O_APPEND|O_CREAT|0o600, single os.write, os.fsync, no try/except around write/fsync — propagation IS the defence (DEC-011 of #4 / DEC-016 of #6 / DEC-006 of #7). audit_path routed through warehouse._path_safety.canonicalise_path at entry; symlink-escape wraps as GradeAuditWriteError(cause=...). OSError/PermissionError on the open/write propagate raw. _build_grade_event is the single GradeEvent construction site (US-009 will add the 6th AST scan to gate this). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.4: load_grade_config + signalforge.yml grade: namespace Inner GradeConfig (extra=forbid) with DEC-023..DEC-027 defaults: model, cache_ttl="1h", max_output_tokens=256, max_retries_429/5xx/conn, total_budget_seconds=300, min_pass_rate=0.7, min_mean_score=0.5, rubric (None=use DEFAULT_RUBRIC), fail_on_below_threshold=False. Outer _GradeConfigFile (extra=ignore at top level) silently tolerates sibling stages (safety:, llm:, prune:). Typos in grade: block fail loud via Pydantic ValidationError, wrapped in GradeConfigError. Resolution order mirrors load_prune_config: explicit path > project_dir/ signalforge.yml > defaults. Missing explicit path raises GradeConfigError; empty / non-mapping / no grade-block YAML returns defaults. rubric override replaces wholesale; duplicate ids re-raise GradeRubricError through @model_validator. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.8: grade_artifacts orchestrator + sidecar JSON + fake helper grade_artifacts(model, candidate, prune_result, *, rubric, config, audit_path, sidecar_path, client, project_dir) -> GradingReport. Sequential criterion-outer iteration over (criterion, artifact) pairs; one call_anthropic per pair (DEC-004); fail-closed JSONL per call; sidecar JSON end-of-run via write_grading_report (DEC-006/012). _artifact_id_for canonical formatter for the 6 DEC-009 dotted-path shapes; _model_test_args_hash disambiguates colliding model-level test types. _stable_artifact_pairs / _iterate_artifacts yield stable order for reproducibility. Total wall-clock budget (DEC-023) cuts off remaining pairs as degraded GradingResult(score=None, passed=False, reasoning="grade budget exceeded ...") with matching GradeEvent records. Per-call retry exhaustion / parser failure also degrade gracefully (DEC-015) — the run never aborts mid-loop except on audit-write failure. Whole-run pre-flight envelope-breach scan: any artifact payload containing </ARTIFACT> raises GradePromptEnvelopeBreachError before any LLM call. run_id (uuid4 hex per DEC-020) ties every JSONL record to the sidecar. prompt_version_template + criterion_prompt_hash + rubric_hash carried on every GradeEvent (DEC-014, DEC-019, DEC-010 of #5). Module-level _sleep alias for deterministic budget tests (DEC-023 / DEC-019 of #6 / DEC-004 of #5). One INFO log per invocation with lazy-format JSON; no f-string interpolation in _LOGGER calls. Public re-exports complete in signalforge.grade.__init__.py. tests/grade/_fake.py::expect_grade_responses helper wraps FakeAnthropicClient (DEC-021). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.9: 6th AST scan (GradeEvent) + logger gate extension tests/test_audit_completeness.py extends with the sixth AST scan gating Call(func=Name(id='GradeEvent')) to signalforge.grade.audit only, with sanity check that >=1 construction site exists in audit.py (catches accidental rename-without-update). Mirrors the 5th scan (PruneEvent) verbatim per DEC-018 of #6. tests/llm/test_logger_grep_gate.py adds _GRADE_DIR to its directory list — the gate now scans signalforge.{llm,draft,prune,grade} for _LOGGER.<level>(f"...) calls. Mirrors DEC-017 of #6. Manually verified both gates fire on intentional violations: - Temporarily added a GradeEvent(...) construction in engine.py; test_grade_event_construction_only_in_grade_audit_module FAILED. - Temporarily added _LOGGER.info(f"...") in engine.py; test_no_f_string_logger_calls_*_modules FAILED. Both reverted before commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.10: Real-API smoke test for grade_artifacts (gated) tests/grade/test_smoke_real_api.py mirrors tests/draft/test_smoke_real_api.py: pytestmark = pytest.mark.anthropic so the default `pytest` invocation deselects it (per pyproject addopts -m 'not anthropic'). Skips when ANTHROPIC_API_KEY is unset. Builds a minimal CandidateSchema (1 column, 1 not_null test) and a single-criterion rubric (clarity). One real grade_artifacts run against this fixture issues 5 LLM calls (~$0.025 on Sonnet 4.6) and asserts SHAPE only (no specific scores or passed outcomes -- LLM output is non-deterministic). Verifies sidecar JSON + JSONL audit are written and parse cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.11: docs/grade-ops.md + cost-transparency note Operational reference mirroring docs/prune-ops.md structure: public API surface, signalforge.yml grade: block schema with worked example, decision matrix (score [0.0, 1.0] + aggregate AND-thresholds), audit/ sidecar shapes (JSONL fail-closed per call + JSON end-of-run sidecar), reproducibility hash fields, failure-mode error-class table, regen instructions for fixtures. Cost guidance per DEC-014: per-criterion fan-out (DEC-004) costs ~$0.18/model on Sonnet 4.6 vs ~$0.05/model batched (Q4=A in plan); the 3.4x trade buys per-criterion retry isolation + v0.2 prompt-tuning headroom. v0.2 will offer batched-criteria as opt-in. example_config.yml extracted verbatim from the doc; tests/grade/ test_config.py extended with a round-trip test that loads the fixture through load_grade_config and asserts the locked DEC-023..DEC-027 defaults are populated. Doc + fixture stay in sync via the round-trip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.12: Quality Gate — fix bugs from 4 code-review passes Pass 1 (security/safety): - engine.grade_artifacts now canonicalises audit_path/sidecar_path against resolved_project_dir BEFORE handing off to writers (mirrors prune.engine). Without this, the writers' own canonicalise_path call derived project_dir from the path itself (audit_path.parent.parent), which neutered the symlink-escape gate for caller-supplied paths (e.g. audit_path=/tmp/grade.jsonl → project_dir="/" → any escape passes containment). - Added prune_result.model_unique_id == model.unique_id assertion at engine entry; raises GradeError on mismatch. Prevents stale prune result from silently driving the no-redundant criterion. Pass 2 (correctness/edge cases): - _test_args_hashes generalised from model-only to also disambiguate column-scope test collisions (e.g. two accepted_values tests on the same column with different values lists). _artifact_id_for now accepts args_hash on the column-scope branch; extract_artifact_text parses the 5-part dotted form. Without this fix, JSONL records on the same (run_id, artifact_id, criterion_id) triple would collide and the diff renderer (#8) couldn't distinguish them. Pass 3 (API design): - Documented GradeBudgetExceededError as v0.2-reserved (currently never raised; engine always degrades). - Documented GradeThresholds as v0.2-reserved (exported but not consumed in v0.1 production code; GradeConfig carries the flat fields). - Documented fail_on_below_threshold as v0.1 no-op (reserved for v0.2 CLI exit-code enforcement). Pass 4 (test quality): - Added regression tests for the QG pass 1 + pass 2 fixes: - test_grade_artifacts_rejects_prune_result_for_different_model - test_grade_artifacts_user_supplied_audit_path_outside_project_tree_rejected - test_grade_artifacts_user_supplied_sidecar_path_outside_project_tree_rejected - test_artifact_id_for_column_scope_collision_args_hash_disambiguates - test_artifact_id_for_column_scope_unique_test_no_args_hash CodeRabbit review: not run — coderabbit:code-review skill is not available in this session. Skill-based CodeRabbit is documented as optional in /ralph-run Step 3b ("Run CodeRabbit review using the coderabbit:code-review skill (if available)"). Operator can invoke manually on the PR if desired. Validation gate: 955 passed (up from 952 pre-QG; +3 net new regression tests after QG pass 4), 10 deselected, ruff clean, pyright 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-dgv.13: Patterns & Memory — grade-layer.md + CLAUDE.md .claude/rules/grade-layer.md distils every load-bearing pattern from issue #7: conservative score-and-degrade taxonomy (DEC-002/015), fail-closed JSONL + sidecar with engine-level path canonicalisation (DEC-006/012, post-QG fix), <ARTIFACT> envelope + whole-run pre-flight breach guard (DEC-008/013), per-(artifact × criterion) sequential calls with cached rubric block (DEC-004/027), reproducibility hash fields (DEC-010/019), canonical artifact_id format with column-scope collision disambiguator (DEC-009 + post-QG fix), single GradeEvent construction seam + 6th AST scan (DEC-029), ANSI-safe logger gate extension (4 dirs now), prune_result.model_unique_id boundary check, custom __repr__ on result-shaped models, drift detectors, API alignment, signalforge.yml grade: namespace, and v0.2 forward-compat reservations. CLAUDE.md updated: Repository status now lists 7 issues shipped (added the #7 grader entry with the public surface highlight); Public API surface section adds the grader's exports. No production code touched. Validation: 955 passed, ruff clean, pyright 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 7: Address PR review feedback (Copilot + CodeRabbit) 15 items fixed across 9 files; 3 marked false positive (see PR comment for justifications). **Code fixes** - engine.py: exact duplicate tests now get an ordinal suffix (`<hash>:<n>`) on top of the args_hash collision disambiguator, so semantically identical tests produce distinct artifact_ids and JSONL records don't collide on the (run_id, artifact_id, criterion_id) triple. _test_args_hashes refactored to share the assignment logic across model-level and column-level scopes. - prompts.py extract_artifact_text: when an artifact_id carries a supplied args_hash, the resolver now re-runs _model_test_args_hash (lazy-imported from engine to avoid module-load cycle) and filters matches by hash, restoring the orchestrator-formatter→resolver round-trip claim. Strips the ":<n>" ordinal suffix when present (exact duplicates have identical rationale, so any match is correct on the read path). - config.py / rubric.py: _bounded_unit and _reject_out_of_range now reject NaN / Infinity via math.isfinite up-front (bare comparisons silently accept NaN). - rubric.py Criterion: validator now rejects embedded NUL bytes (\\x00) to keep criterion_prompt_hash's NUL domain-separator contract honest. - test_smoke_real_api.py: skip-gate now treats empty/whitespace-only ANTHROPIC_API_KEY as unset (avoids a noisy live-call auth failure). **Doc fixes** - errors.py GradeRubricError: remediation rewritten to reference the actual `grade.rubric` block + `id`/`criterion` fields (the old text mentioned `rubric_path`/`description`/`weight` which never shipped). - errors.py GradeAuditWriteError: remediation now mentions BOTH the JSONL audit and the sidecar JSON paths (one error class, two writers). - errors.py GradeAuditRecordTooLargeError: remediation branches on the cap (4 KB POSIX-atomic-append for JSONL vs 1 MB sidecar cap). - docs/grade-ops.md: escape `#2` at line start (markdownlint MD018); add `text` language tag to the prompt-injection example fence (markdownlint MD040). - plans/super/7-quality-grader.md: add `python` language tag to the DEC-016 default-rubric block fence (markdownlint MD040). **Regression tests added (3)** - test_extract_artifact_text_filters_by_args_hash_for_model_collision - test_extract_artifact_text_filters_by_args_hash_for_column_collision - test_stable_artifact_pairs_exact_duplicate_tests_get_ordinal_suffix **False positives (documented)** - _fake.py permissive matchers — intentional for the v0.1 fake's call-order/count contract. - __init__.py docstring — CodeRabbit's "scaffold-only" critique was read against a stale base; the current docstring describes the full API surface. - audit.py __all__ exports underscore-prefixed names — intentional; documents the test-importable surface (size caps + the single GradeEvent construction seam blessed by the AST audit-completeness scan); underscore prefix remains the public-API marker per repo convention. Sibling safety/draft/prune audit modules follow the same pattern. Validation: 958 passed (up from 955; +3 net new regression tests), 10 deselected, ruff clean, pyright 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 8: Diff renderer (plan) Super plan for #8 — kept/dropped table + unified schema.yml diff. 14 stories (12 implementation + Quality Gate + Patterns & Memory), 21 decisions, architecture review across security / performance / data-model+API / observability / testing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 8: Mark plan phase published; link PR #25 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 8: Devolve plan; record beads manifest (epic + 14 tasks) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.4: Diff safety helpers — _ansi_safety, _markdown_safety (US-004) Two leaf modules for the diff renderer (#8): - `signalforge.diff._ansi_safety.strip_ansi_escapes` — regex-based stripper for ANSI CSI escape sequences (DEC-007). Defends the Markdown sink against terminal-control injection from upstream manifest fields, LLM-drafted artifact text, and prune/grade reasons. - `signalforge.diff._markdown_safety.escape_markdown_scalar` — escapes backtick/pipe/backslash; HTML-entity-encodes pipe and row-breaking control chars in table-cell mode (DEC-008). Backslash is processed first so subsequent escapes can't be unwound by a crafted trailing-backslash. Tests cover bare CSI, color codes, reset, compound SGR, cursor movement, no-escape passthrough, idempotence, empty string for ANSI; backtick/pipe/backslash escaping, table-cell entity encoding, control-char encoding, idempotence, empty string for Markdown. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.1: Diff error hierarchy (US-001) Seven-class typed exception hierarchy under signalforge.diff.errors, mirroring the safety / draft / prune / grade precedent: every error carries a class-level default_remediation that __str__ renders on a separate ↳ Remediation: line, and every user-supplied string flowing into a message routes through repr() (DEC-022 of #6) so adversarial input cannot inject ANSI escapes or control characters into log viewers. Classes: DiffError (base) + DiffCandidateModelMismatchError, DiffPruneResultModelMismatchError, DiffGradingReportModelMismatchError (DEC-002 boundary checks at orchestrator entry); DiffInputTooLargeError (DEC-006 existing-schema YAML byte cap); DiffSidecarRecordTooLargeError (DEC-009 sidecar size cap); DiffSidecarWriteError (fail-closed sidecar-write seam wrapping underlying I/O cause via __cause__). 24 tests covering default_remediation rendering, repr() escaping for ANSI-bearing inputs, subclass-of relationships, and field exposure. Subpackage __init__.py ships only the error hierarchy in US-001; later stories (US-002 result models, US-003 config, US-010 orchestrator) extend the public surface. Traces to plans/super/8-diff-renderer.md US-001 (DEC-002, DEC-006, DEC-009). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.5: Canonical YAML emitter (US-005) Add signalforge.diff._emitter.emit_proposed_yaml(candidate, prune_result) that filters tests to PruneDecision.decision == "kept", preserves column declaration order, and sorts tests within each column by (type, args_hash) for deterministic output. Mirrors the grade layer's args_hash convention (blake2b-4 of canonical-sorted JSON of test args, DEC-009) so the emitter, grader, and diff renderer agree on test identity. yaml.safe_dump invoked with sort_keys=False, default_flow_style=False, width=4096, allow_unicode=True. AR-9 round-trip test verifies edge-case descriptions ('---', '!tag', triple-backticks, embedded newlines, leading quote/pipe/gt, unicode) survive emit -> yaml.safe_load byte-identical. Leaf module — depends only on existing signalforge.draft and signalforge.prune model types. No __init__.py created (US-001 owns it). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.6: Diff artifact-id formatter (US-006) Add src/signalforge/diff/_artifact_id.py with byte-equal mirror of signalforge.grade.engine._artifact_id_for. Six dotted-path shapes (column.<col>.{description,rationale}, model.{description,rationale}, test.column.<col>.<type>[.<args_hash>], test.model.<type>[.<args_hash>]) plus _model_test_args_hash and compute_args_hashes helpers. Cross-stage parity is load-bearing: the diff renderer joins grade- sidecar JSON to its rendered diff via (run_id, artifact_id, criterion_id); a single shape disagreement would silently drop grade rows. tests/diff/test_artifact_id.py is the documented single allowed cross-stage import seam — production diff code must not import from signalforge.grade at runtime. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.2: Diff result models (US-002) Adds DiffEntry and DiffReport read-back-stable Pydantic v2 models to src/signalforge/diff/models.py per US-002 of issue #8. Both models are frozen=True, extra="ignore" with custom __repr__ per DEC-020 that omits prose/diff text fields. DiffEntry.tier is a Literal["kept","dropped", "flagged"] per DEC-012; DiffReport carries reproducibility-hash fields (candidate_hash, prune_result_hash, grading_report_hash) per DEC-016. Drift detector at tests/diff/test_models.py mirrors the prune / grade precedent: StrictDiffEntry / StrictDiffReport extra="forbid" mirrors validated against tests/fixtures/diff/diff_report_v1.json (which exercises all three Tier values), plus field-set parity and extra="forbid" sanity-floor checks. Public-API re-exports deferred to US-012 per the bead instructions — diff/__init__.py is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.3: DiffConfig + load_diff_config (US-003) Introduces the `diff:` top-level namespace in `signalforge.yml` per DEC-010 of plans/super/8-diff-renderer.md. `DiffConfig(extra="forbid", frozen=True)` carries the nine locked knobs (context_lines, max_why_chars, narrow_terminal_threshold, markdown_max_diff_chars, existing_schema_size_limit_bytes, existing_schema_warn_at_bytes, sidecar_size_limit_bytes, render_kind, respect_no_color_env); `_DiffConfigFile(extra="ignore")` outer wrapper silently tolerates sibling stage namespaces (`safety:`, `llm:`, `prune:`, `grade:`). `load_diff_config(project_dir, path=None) -> DiffConfig` matches the prior-stage signatures (load_grade_config / load_prune_config / load_draft_config / load_safety_config) verbatim. Numeric knobs route through a single positive-integer validator — zero/negative caps would silently disable the DEC-006/DEC-009 protections or render an empty table. The literal `render_kind` discriminator catches unknown render targets at config-load time. Per US-003 task scope, the loader raises the base `DiffError` (with remediation) for config-load failures rather than introducing a new `DiffConfigError` subclass; the seven-class hierarchy from US-001 stays intact, and a future `DiffConfigError` is a clean v0.2 refinement that won't break imports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.8: AnsiRenderer (US-008) Adds the Renderer ABC + AnsiRenderer concrete in src/signalforge/diff/_renderers.py per US-008 of plans/super/8-diff-renderer.md. * DEC-007 — strip_ansi_escapes runs UNCONDITIONALLY on every user-content field (artifact_id, why, drop_reason, test_type, unified_diff body, model_unique_id) BEFORE the renderer's own colour codes are emitted. Defence-in-depth: a malicious "\x1b[31mEVIL\x1b[0m" in upstream content renders as the literal text "EVIL" in BOTH coloured and non-coloured output. * DEC-013 — narrow-TTY compact mode triggers when the effective terminal width is below DiffConfig.narrow_terminal_threshold (60 by default). Compact mode drops the WHY column from the table and emits each entry's why as an indented "└─ ..." follow-up line. * DEC-021 — six-step colour-precedence chain documented as early-returns in _should_emit_color: respect_no_color_env=False > force_color=False > force_color=True > FORCE_COLOR env > NO_COLOR env > sys.stdout.isatty(). * Module structured so US-009 can append MarkdownRenderer without restructuring; only the ABC + AnsiRenderer ship in this ticket. * No _LOGGER calls; renderer does no I/O (returns string). Tests at tests/diff/test_renderers.py cover the ABC contract, wide-TTY 6-column table, narrow-TTY compact mode, all six DEC-021 precedence positions individually, and unconditional ANSI stripping in both coloured and non-coloured output across the artifact_id, why, follow-up why, model_unique_id, and unified_diff body fields. 25 tests, all passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.7: Diff sidecar writer (US-007) Add `signalforge.diff._sidecar` — fail-closed JSON sidecar writer for the diff renderer (DEC-009). Mirrors `grade-layer.md` DEC-006/012 verbatim: single-document overwrite, ``O_WRONLY | O_CREAT | O_TRUNC | 0o600``, single ``os.write`` (looped on short returns), ``os.fsync``, descriptor-release-only ``try / finally``. No ``except`` handler around write/fsync — the propagation IS the defence. * ``_DIFF_SIDECAR_RECORD_LIMIT_BYTES = 10_000_000`` (10 MB; an order of magnitude above the grade sidecar's 1 MB cap because diff text is larger by nature). Pre-write size check raises ``DiffSidecarRecordTooLargeError`` BEFORE any ``os.open`` so an oversize payload leaves no on-disk artefact. * Symlink-hardened path canonicalisation at writer entry via ``signalforge.warehouse._path_safety.canonicalise_path``. Containment is gated against the **caller-supplied** ``project_dir``, not a derivation of ``sidecar_path`` (mirrors grade's post-QG fix). Failures wrap as ``DiffSidecarWriteError(cause=...)``. * AST defence test in ``tests/diff/test_sidecar.py`` walks the module's syntax tree and asserts: (a) exactly one ``ast.Try`` block guards the ``os.write`` / ``os.fsync`` syscalls (the descriptor-release ``try / finally``), and (b) that block has zero ``except`` handlers. An accidental ``try / except OSError`` around the write/fsync would silently swallow the exact failure mode the fail-closed pattern exists to surface. Test coverage (13 cases): happy-path round-trip via ``json.loads``, ``0o600`` mode bits, parent-directory creation, ``O_TRUNC`` overwrite semantics, single ``fsync``, oversize pre-flight (no on-disk artefact), symlink-escape rejection (containment gate against caller-supplied ``project_dir``), absolute-path-outside-project rejection, symlink-loop rejection, short-write loop covering the full payload, zero-byte ``write`` raises ``OSError``, ``OSError`` propagation, and the AST defence above. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.9: MarkdownRenderer (US-009) Extends `signalforge.diff._renderers` with `MarkdownRenderer`: - GitHub-flavored Markdown output (heading + count summary + GFM pipe-table + fenced ```diff block). - Per-cell escaping via `escape_markdown_scalar(in_table_cell=True)` per DEC-008; pipe / backtick / backslash / row-breaking control chars are entity-encoded inside cells, while the fenced ```diff block carries raw upstream content (the fence is the defence). - DEC-005 truncation: when `unified_diff` exceeds `config.markdown_max_diff_chars` (default 60_000), the body is truncated at the last complete `@@` hunk boundary; a footer `... (N more lines truncated — see <project_dir>/.signalforge/diff.json for full diff)` lives INSIDE the fenced block. Falls back to a line-boundary cut for hunk-less bodies. - DEC-007 strip-then-escape composes: `strip_ansi_escapes` runs unconditionally on every user-content field (model_unique_id, artifact_id, why, drop_reason, test_type, diff body) BEFORE markdown-escaping or fence-wrapping. - Empty `unified_diff` suppresses the diff block entirely; empty entries tuple emits `_(no candidate artifacts)_`. - `project_dir` constructor kwarg renders into the truncation footer (placeholder `<project_dir>` when None — stable for snapshots). Tests cover: ABC subclass-of, pipe-table well-formedness, fenced diff passthrough vs. table-cell escaping (the same pipe / header content is encoded in cells but raw inside the fence), truncation at last hunk + dropped-line count + fence-internal footer placement, project_dir rendering, ANSI strip in cells + diff body, and the empty-diff / empty-entries fallbacks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.10: Diff orchestrator render_diff + JsonRenderer (US-010) Wires every prior story (errors US-001, models US-002, config US-003, safety US-004, emitter US-005, artifact-id US-006, sidecar US-007, renderers US-008/009) into one public seam: render_diff(model, candidate, prune_result, *, grading_report, existing_schema, config, output_path, sidecar_path, project_dir) -> DiffReport. Boundary checks (DEC-002) raise BEFORE any other work; existing_schema size cap (DEC-006) + soft-warn (DEC-014) gate yaml.safe_load; symlink-hardened path canonicalisation against project_dir for both output_path and sidecar_path mirrors the grade-engine post-QG fix verbatim. Reproducibility hashes (DEC-016) computed via canonical-sort blake2b-8 of each input. Single INFO log at happy-path end with lazy-format json.dumps payload (DEC-015). JsonRenderer added to _renderers.py — model_dump_json(indent=2, by_alias=True), stateless. 19 new tests cover the boundary-check trio, the size-cap + soft-warn pair, renderer dispatch across all three concretes, JsonRenderer round-trip, sidecar write, output_path / sidecar_path symlink containment, INFO log shape, and an end-to-end happy path that asserts kept/dropped/flagged tier assignment and DEC-016 hash presence. * bd_1-scaffolding-htq.11: Diff snapshot fixtures + regenerate.sh + drift detector (US-011) Commits the 10-case fixture matrix (DEC-017 of #8) under tests/fixtures/diff/, the deterministic snapshot-input builder at tests/diff/_snapshot_inputs.py, the regenerate script tests/fixtures/diff/regenerate.sh (mirrors tests/fixtures/regenerate.sh shape), the byte-equality snapshot tests at tests/diff/test_snapshot_fixtures.py, and the schema-drift detector tests/diff/test_drift_detector.py with StrictDiffReport / StrictDiffEntry mirrors validated against diff_report_v1.json and the new diff_entry_v1.json fixture. The 11 fixture artefacts on disk cover: full_with_grade.{ansi,md,json} — happy path, three surfaces. no_existing_schema.ansi — /dev/null source for unified diff. kept_only.ansi — every artifact tier=kept. dropped_only.ansi — every artifact tier=dropped. no_grading_report.ansi — score columns null; no flagged tier. plain_no_color.txt — ANSI surface w/ NO_COLOR=1. narrow_terminal.ansi — 40-col TTY (DEC-013 compact mode). injection_payloads.{ansi,md} — adversarial content (DEC-007/008). Static checks on the plain_no_color and injection_payloads fixtures pin DEC-007 (strip user-content ANSI escapes UNCONDITIONALLY) and DEC-021 (NO_COLOR + force_color=False produces zero ANSI escapes). The drift detector ships the standard four-test pattern (validate fixture, validate each entry, field-set parity, sanity-floor reject unknown field) per the prune / grade precedent. Validation: ruff / pyright / pytest all green; full suite 1178 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.12: Diff public API + docs + logger grep gate (US-012) Export the v0.1 public surface from signalforge.diff per DEC-004 of plans/super/8-diff-renderer.md: render_diff orchestrator, DiffConfig + load_diff_config, DiffReport / DiffEntry / Tier result models, and the seven-class DiffError hierarchy. Concrete renderers (AnsiRenderer, MarkdownRenderer, JsonRenderer) and internal helpers (_emitter, _sidecar, _artifact_id, _ansi_safety, _markdown_safety) stay private — reachable via dotted import for internal callers but absent from __all__. Add docs/diff-ops.md mirroring docs/grade-ops.md as the operational reference: overview, public API surface, configuration block, renderer-kind selection, sidecar JSON schema, reproducibility hash fields, decision matrix, operational notes (symlink hardening, log events, ANSI / Markdown injection escaping), snapshot fixture matrix, debugging, and failure-mode cross-reference. Extend tests/llm/test_logger_grep_gate.py to scan src/signalforge/diff/ as the fifth directory per DEC-019 — the project-wide ANSI-safe lazy-format logger gate now covers llm, draft, prune, grade, and diff. Planted-violation self-check confirms the gate fires on an f-string interpolated _LOGGER call in signalforge.diff.engine. Add tests/diff/test_public_api.py asserting __all__ matches the documented surface, that every public name imports via ``from signalforge.diff import ...``, that concrete renderers and internal helpers stay out of __all__, that the concrete renderers remain reachable via dotted import (the documented escape hatch), and that all six DiffError subclasses inherit from DiffError. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.13: Quality gate — fix bugs from code review Fixes from 4-pass code review on the diff renderer epic (#8): 1. Inverted soft-warn / hard-cap defaults in DiffConfig (DEC-014 was dead code). Swap so warn_at_bytes (1MB) < size_limit_bytes (10MB), and add a model_validator that fails loud at config-load time when warn_at >= size_limit. 2. DiffConfig.sidecar_size_limit_bytes was silently ignored — wire it through render_diff to write_sidecar via a new size_limit_bytes kwarg. Adds end-to-end test pinning the orchestrator-level error. 3. Drop dead DiffSidecarRecordTooLargeError catch from the _write_rendered_text exception ladder (output_path branch can't raise that error; only sidecar path can). CodeRabbit skill not available in this session; skipped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.14: Diff renderer patterns + CLAUDE.md update (US-014) Distil DEC-001..DEC-021 of the diff renderer (#8) into .claude/rules/diff-renderer.md, mirroring grade-layer.md section structure verbatim. Bake the post-QG-fix lessons (inverted warn/limit defaults; sidecar_size_limit_bytes wiring; dead exception catch) and the load-bearing rules (tier classification + no-grading degrade, fail-closed sidecar, ANSI strip unconditional, Markdown hunk-boundary truncation, three boundary checks at orchestrator entry, reproducibility hashes, drift detector, logger grep gate fifth dir). Update CLAUDE.md Repository status to "Eight issues shipped" with a new bullet for #8 matching the prior format; extend the Public API surface bullet list and Internals listing; remove "diff renderer #8" from the remaining-feature-work line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.13: Address PR #25 review feedback (worker B — engine.py + design Q1/Q3) Six fixes to the diff orchestrator from PR #25 code review: - #1: wrap mkdir in both output_path and sidecar_path branches with try/except → raise DiffSidecarWriteError(cause=...). A project_dir whose parent is an existing FILE no longer leaks an untyped OSError out of the diff layer. - #2: doc artifacts (column/model description/rationale) ALWAYS emit a DiffEntry per Architectural Commitment #5. When grading is absent or no matching result exists, the row gets tier="kept", score=None, passed=None, why="kept (no grading)". Prune-only runs now surface every present description text in the kept/dropped/flagged table. - #3: flagged-tier why now reflects the GRADING reason, not the prune decision why. Format: "failed grading: <criterion_id> — <reasoning>" (truncated to max_why_chars). The first failing criterion drives it for determinism. - #4: replace id(decision.test) lookup with structural keying so the args_hash join survives JSON-rehydration of the prune result. Cross-stage parity with the grade engine artifact_ids is preserved (post-QG fix to enable diff↔grade-sidecar joins after the prune result has been persisted+reloaded). - #5 (Q1=A): default sidecar_path to <project_dir>/.signalforge/ diff.json when write_sidecar=True (default). New write_sidecar kwarg disables the sidecar entirely. Makes the diff sidecar an always-on durable record by default, mirroring grade/prune audit precedent. - #6 (Q3=A): skip renderer.render(report) entirely when there's no consumer (no output_path AND write_sidecar=False / sidecar serialises the report directly). Move duration_seconds capture to immediately before the INFO log so it reflects the full wall-clock including renderer + writes. Tests: 8 new regression tests added; one existing test updated for the sidecar default change; one existing assertion updated for the doc-row emission count change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.13: Address PR #25 review feedback (worker A — non-engine fixes) Code fixes: - _ansi_safety: broaden CSI regex to cover the full final-byte range (@-~), not just letters; add tests for tilde-terminated key sequences (`\x1b[3~`) and bracketed-paste markers (`\x1b[200~`). - _markdown_safety: HTML-entity-encode `&`, `<`, `>` before any Markdown escape (defence against `<script>`, `</details>`, `<img src=x>` smuggling). Order matters: `&` first to avoid double-encoding subsequent escape entities. - _renderers (MarkdownRenderer): four bug fixes — (a) dropped_line_count off-by-one when body ends with `\n`; (b) the WHOLE rendered diff block now fits under `markdown_max_diff_chars` (subtract fence + footer overhead before truncating); (c) when the first hunk alone exceeds the cap, emit only the `---`/`+++` file-header lines rather than a mid-hunk character cut (which would produce malformed unified-diff output); (d) dynamic fence length: scan the body for the longest backtick run and pick `max(3, longest_run + 1)` so a YAML payload containing literal triple-backticks cannot close the outer fence prematurely. - config: wrap `read_text` in try/except for OSError / IsADirectoryError / PermissionError; symlink-harden the resolved config path via `signalforge.warehouse._path_safety.canonicalise_path` before any read (mirrors the orchestrator-level treatment of `output_path` / `sidecar_path`). Docs / plan fixes: - errors.py: `DiffInputTooLargeError.default_remediation` now matches production's 10 MB cap (not the obsolete 1 MB literal). - docs/diff-ops.md: every reference to `existing_schema_size_limit_bytes` (10 MB) and `existing_schema_warn_at_bytes` (1 MB) defaults consistent; fenced log-output blocks tagged as `text` (MD040); soft-warn prose corrected ("warn 1 MB fires below hard cap 10 MB"). - .claude/rules/diff-renderer.md: tagged the truncation-footer fence as `text`. - plans/super/8-diff-renderer.md: DEC-006 / AR-5 / AR-11 / AR-13 / the `DiffConfig` snippet now carry the post-QG-fix defaults (10 MB hard cap, 1 MB soft warn — original plan had them inverted before commit c3dc6ab); validation command snippet now prefixes `pip install -e ".[dev]"`; MD038 inline-code-span spaces removed; duplicate `## Beads Manifest` heading dropped; truncation-footer fence tagged as `text`. Tests: - New regression tests in tests/diff/test_ansi_safety.py (3), test_markdown_safety.py (7), test_renderers.py (5) covering every code fix above. - tests/diff/test_config.py: regression tests for IsADirectoryError wrap and the symlink-escape rejection path. Fixtures: - tests/fixtures/diff/injection_payloads.md regenerated via `tests/fixtures/diff/regenerate.sh` to capture the new HTML-escaped `</details>` rendering of the hostile input. Validation: `ruff check .`, `ruff format --check .`, `pyright`, `pytest` all green (1222 passed). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-htq.13: Address PR #25 round 2 review feedback (CodeRabbit on second pass) Six fixes from the second-pass CodeRabbit review: 1. engine.py — exact-duplicate structural-key collapse. Replace the ``dict[_StructuralKey, str | None]`` that lost duplicate args_hash entries (last-assignment-wins) with a per-key queue ``dict[_StructuralKey, list[str | None]]``. ``_resolve_test_artifact_id`` now pops from the front of the queue per-decision so two byte-identical ``CandidateTest`` instances in ``prune_result.decisions`` get distinct artifact_ids matching grade engine's id()-keyed ordinal suffixes. Iteration order in the queue matches ``signalforge.prune.engine._iter_candidate_tests`` (columns first, then model-level) so positional consumption aligns with the prune walk. 2. engine.py — ``DiffReport.duration_seconds`` refresh. Construct the report with placeholder ``0.0``, refresh via ``model_copy`` after rendering and the optional output_path write but BEFORE the sidecar write, so the persisted JSON, returned report, and INFO log all carry the same wall-clock value. The sidecar write block moved below the refresh. 3. tests/diff/test_engine.py — caplog scoping. Three call sites that filtered by ``levelname`` only now also gate on ``rec.name == "signalforge.diff.engine"`` so unrelated logger records can't false-positive the assertions. 4. docs/diff-ops.md — sidecar API/default sync. The ``render_diff`` signature now shows the ``write_sidecar=True`` kwarg; the sidecar section documents the on-by-default semantics with the ``<project_dir>/.signalforge/diff.json`` path and the ``write_sidecar=False`` opt-out. 5. .claude/rules/diff-renderer.md — sidecar default semantics. The rule prose now reflects the post-Q1 default-on shape. 6. .claude/rules/diff-renderer.md — ANSI strip regex contract. The documented regex now matches the broadened ECMA-48 / ISO 6429 CSI grammar (``\x1b\[[0-?]*[ -/]*[@-~]``) shipped during US-014. Tests added (3): exact-duplicate not_null, exact-duplicate accepted_values, and a cross-stage parity test that compares diff-side artifact_ids against grade engine's ``_artifact_id_for`` output for the duplicate scenario. Validation: ruff clean, ruff format clean, pyright clean, pytest 1225 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 9: CLI entrypoint super plan Capture the design for the v0.1 CLI ticket: discovery, architecture review, 20 decisions (DEC-001..DEC-020), and 12 right-sized stories ready for Ralph. Notable: SQ-03=C was overridden during architecture review to a hybrid that graduates GradeConfig.fail_on_below_threshold (the v0.2 reservation from #7) inside the grade layer instead of shipping a parallel `cli:` config block. One knob, one owner; saves ~250 lines of config plumbing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Update plan meta — PR #26 draft published Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Plan refinements — DEC-021..DEC-027 (post-publish review) Resolve seven issues surfaced by an external review pass against the published plan. All seven verified against the actual codebase before applying: - DEC-021 (US-002): GradeBelowThresholdError raise lands AFTER write_grading_report and BEFORE return, preserving the durable sidecar hand-off on threshold-fail. New ordering test pins it. - DEC-022 (US-001): render_to_text uses the real _build_renderer dispatcher; DiffReport carries no config_used field, so the helper defaults to DiffConfig() rather than reaching into the report. - DEC-023 (US-007): --no-color sets NO_COLOR=1 env (DiffConfig has no force_color field; only respect_no_color_env=True). - DEC-024 (US-008): AST-scan target verified — every typed exception in the project lives in an errors.py module. Explicit exclude list is the 9 abstract base classes. - DEC-025 (US-005): Add test_generate_calls_stages_in_documented_order pinning safety → draft → prune → grade → diff. - DEC-026 (US-007): Drop hardcoded duration hints from progress lines; replace with live values + post-hoc "done in Xs" line. - DEC-027 (US-005): --project-dir override is an absolute assertion; walk-up only applies to unflagged invocations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Devolve plan to beads — phase devolved Phase 7 of /super-plan: convert the 12 stories from the detailed breakdown into a beads task graph so Ralph can pick them up. Created: 1 epic (bd_1-scaffolding-9vj) + 12 child tasks (bd_1-scaffolding-9vj.1 through .12). Wired 31 dependency edges per the documented dependency graph. Initial ready set: US-001 and US-002 (no blockers; can run in parallel). Plan phase: published → devolved. Beads Manifest section populated with the bead IDs and dependency table. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-001 — Public-surface plumbing (re-exports + render_to_text) Re-exports 8 typed exceptions from signalforge.draft, 1 from signalforge.llm so the CLI can tier-map them via the public surface. Adds signalforge.diff.render_to_text(report, *, config=None, project_dir=None) -> str — internal wrapper around _build_renderer + renderer.render(report) that returns the same bytes render_diff would have written to output_path. Traces to DEC-013, DEC-015, DEC-022. bd_1-scaffolding-9vj.1 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-002 — Grade-layer graduation: GradeBelowThresholdError Graduates GradeConfig.fail_on_below_threshold from v0.1 no-op to v0.1 wiring. New GradeBelowThresholdError raised inside grade_artifacts AFTER write_grading_report returns and BEFORE return — sidecar JSON lands on disk first so the operator has a durable hand-off for diagnosis (DEC-021 ordering invariant pinned by test_grade_below_threshold_writes_sidecar_before_raising). Doc cascade: grade-layer.md v0.2-reservations block updated; GradeConfig.fail_on_below_threshold docstring rewritten; docs/grade-ops.md gains "Threshold-fail behaviour" section. Traces to DEC-011, DEC-021. bd_1-scaffolding-9vj.2 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-003 — CLI scaffold + version subcommand + logger grep-gate ext Lays the foundation for the signalforge CLI: pyproject [project.scripts] entry; cli/__init__.py with main(argv) + argparse parser + version flag; cli/_helpers.py with canonicalise_user_path, setup_logging, format_error_to_stderr, map_exception_to_exit_code, _safe_excepthook; cli/version.py with --version + version subcommand sharing one string; cli/errors.py with CliError / CliPathError / CliInputError. Logger grep gate now scans 6 dirs (cli/ added). Smoke + dispatch tests use in-process main(argv); no subprocess (US-009 owns that). Traces to DEC-007, DEC-008, DEC-013, DEC-016, DEC-017. bd_1-scaffolding-9vj.3 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-005 — cli/generate.py core orchestration signalforge generate <model> wires manifest → safety → draft → prune → grade → diff. Project-root resolution: walk-up from cwd, --project-dir override is an absolute assertion (DEC-027). --manifest / --profiles-dir flow through canonicalise_user_path. Every typed exception caught at cmd_generate boundary, mapped via _helpers.map_exception_to_exit_code, formatted via _helpers.format_error_to_stderr; no traceback ever leaks. Stage-order test pins the documented safety → draft → prune → grade → diff pipeline (DEC-025). Knob flags (--mode/--min-score/--write/...) land in US-006; observability flags (--quiet/--verbose/...) land in US-007. Traces to DEC-001, DEC-007, DEC-008, DEC-011, DEC-013, DEC-015, DEC-016, DEC-017, DEC-025, DEC-027. bd_1-scaffolding-9vj.5 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-006 — Generate flags --mode, --min-score, --write/--dry-run, --format Layers five runtime-knob flags onto US-005's signalforge generate: --mode (overrides safety policy via SafetyPolicy.with_mode); --min-score (reporting-only override of grade.min_mean_score; never affects exit code by itself per DEC-004); --write writes proposed schema.yml to disk; --dry-run runs full pipeline and writes nothing (overrides DEC-002's default-on sidecar); --format selects render_kind. --write and --dry-run are mutex at argparse level. Traces to DEC-002, DEC-004, DEC-010, DEC-020. bd_1-scaffolding-9vj.6 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-007 — Generate observability flags + progress lines Layers the UX knobs onto signalforge generate: --quiet suppresses progress and raises log level to WARNING; --verbose raises log level to DEBUG and allows panic-path tracebacks (skips _safe_excepthook install); --no-color sets NO_COLOR=1 env so the AnsiRenderer's existing precedence chain emits plain text (DEC-023 — DiffConfig has no force_color field). Five stderr progress entry lines + five paired 'done in Xs' exit lines per stage; live values only (no hardcoded duration hints per DEC-026). TTY-gated by default (both stderr and stdout must be terminals). Traces to DEC-014, DEC-016, DEC-023, DEC-026. bd_1-scaffolding-9vj.7 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-008 — Exit-code AST scan + parametrized tests 7th AST scan (tests/test_audit_completeness.py) walks every src/signalforge/*/errors.py + src/signalforge/cli/errors.py and asserts every non-abstract *Error class appears in signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE (DEC-024 — explicit 9-class exclude list for the abstract per-stage bases). New tests/cli/test_exit_codes.py parametrizes over every typed exception in the mapping table; for each, patches a stage to raise it, calls main(["generate", ...]), asserts the exit code matches the table value and stderr shape per DEC-008 (single line for tiers 1/3, header + bullets for tier 2). Anchor-contract test pins the tier-2 bullet shape; panic test pins exit 1 + no traceback. Makes the four-tier taxonomy a contract. Traces to DEC-008, DEC-019, DEC-024. bd_1-scaffolding-9vj.8 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-004 — cli/lint.py config-only validator signalforge lint validates the 5 existing signalforge.yml config blocks (safety, llm, prune, grade, diff) without touching warehouse, LLM, or network. Multi-error reporting per DEC-008: zero errors → exit 0 (silent stdout); one error → single ERROR line; multiple errors → header + bullets shape so the operator sees every problem in one run. --config / --project-dir flags flow through canonicalise_user_path; lint runs sub-second on a representative fixture. Traces to DEC-006, DEC-008. bd_1-scaffolding-9vj.4 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-009 — Subprocess-gated CLI smoke test One belt-and-braces test that runs signalforge --version via subprocess.run. Gated behind a cli_subprocess pytest marker so default runs skip it; maintainers run pytest -m cli_subprocess before declaring a CLI PR ready (mirrors the bigquery / anthropic integration-test gates). Catches console-script wiring drift that in-process main(argv) tests cannot. Traces to DEC-018. bd_1-scaffolding-9vj.9 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-010 — Documentation cascade (cli-ops.md + README + cli-layer.md) New docs/cli-ops.md (full CLI reference: subcommands, flags, four-tier exit-code taxonomy, stderr shapes, env vars, project-root discovery, worked example). New .claude/rules/cli-layer.md distilling the rules established by US-001..US-009 (subpackage layout, exit-code taxonomy, stderr shapes, no-traceback rule, path canonicalisation, logger grep gate covers 6 dirs, 7th AST scan, subprocess smoke, progress UX). README updated: Status block bumped to "Nine issues shipped"; quick-start promoted to active voice; new CLI section. v0.2-reservation cascade: grade-layer.md fail_on_below_threshold + diff-renderer.md render_kind / render_to_text both noted as graduated by #9. bd_1-scaffolding-9vj.10 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Quality gate pass 1 — fix --profiles-dir containment-gate bug QG pass 1 caught: --profiles-dir was routed through canonicalise_user_path which contains paths inside project_dir, but the dbt convention places profiles.yml at ~/.dbt (intentionally outside the project tree). Every realistic --profiles-dir value would have exited 1 with CliPathError. The existing test masked the bug by placing the profiles dir inside the project. New test test_generate_profiles_dir_accepts_out_of_tree_path pins the corrected behaviour. Fix: bypass canonicalise_user_path for --profiles-dir; apply expanduser + resolve(strict=False) for symlink-loop safety; the warehouse loader retains its own existence/shape gate on the resolved profiles.yml. bd_1-scaffolding-9vj.11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Quality gate pass 2 — fix --profiles-dir / --no-color help-string accuracy QG pass 2 noted that the help strings claimed "for the duration of the run" but the env mutations actually persist for the process lifetime. Strikes the misleading phrasing on both flags so the help text matches actual behaviour. Restoration semantics deferred to v0.2 when an in-process batch runner makes process-scoped env mutations a real correctness concern. bd_1-scaffolding-9vj.11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Quality gate pass 3 — fix --min-score contract drift (DEC-004) QG pass 3 found that DEC-004 / docs / help text / test name all claimed --min-score "drives the diff renderer's flagged tier", but the code only mutates grade_config.min_mean_score (an aggregate-verdict threshold). The diff layer's flagged tier flips on per-criterion GradingResult.passed, set verbatim by the LLM judge — it never reads min_mean_score. The flag works as implemented; the documented contract was wrong across 5 surfaces (help text, module docstring, code comment, ops doc x2, test name + docstring, plan DEC-004). Aligns the documented semantics with actual behaviour: --min-score is a reporting-only override of the aggregate-verdict threshold consumed by GradingReport.passed (and, opt-in via grade.fail_on_below_threshold, by GradeBelowThresholdError). Per-criterion flagged classification would be a v0.2 feature. Captures the correction as QG-002 in the plan's Refinement Log Session 3 alongside QG-001 (--profiles-dir). bd_1-scaffolding-9vj.11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Quality gate pass 4 — finish docs/cli-ops.md parity (DEC-023) QG pass 4 sign-off pass caught that pass 2's "duration of the run" correction was applied to the help strings in cli/generate.py but missed three parallel mentions in docs/cli-ops.md. Pass 3 set the multi-surface parity bar (help / docstring / comment / ops doc / test name); applying that bar to pass 2's surfaces surfaces three remaining doc-only inconsistencies. Mirror the help-string wording verbatim in --profiles-dir / --no-color / Environment-variables sections of docs/cli-ops.md so the operator sees one consistent phrasing about scope (process environment) instead of the misleading "for the duration of the run" claim. bd_1-scaffolding-9vj.11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: US-012 — Patterns & Memory closeout for the CLI entrypoint ticket CLAUDE.md repo-status bumped to "v0.1 alpha. Nine issues shipped" (matching the README's wording from US-010); new #9 bullet matching the format of the prior 8 bullets covers console-script entry, three subcommands + flag set, walk-up + absolute-assertion project-root resolution, four-tier exit-code taxonomy + 7th AST scan, DEC-021 sidecar-then-raise ordering, render_to_text + 8 exception re-exports, 6-dir logger grep gate. Public-API-surface section gains a CLI entry covering signalforge.cli.main, the CliError hierarchy, the console script, and signalforge.diff.render_to_text (graduated this ticket). .claude/rules/cli-layer.md refined with three new sections: - os.environ mutation pattern (DEC-023): document the deliberate v0.1 lack of try/finally restoration; reserve restoration for a v0.2 in-process batch runner. - _EXCEPTION_TO_EXIT_CODE mapping table convention (DEC-024): identity-keyed dict, MRO walk for subclasses, abstract-base exclude list as the v0.2 seam. - Multi-surface parity rule (QG pass-3 lesson): codify the 5-surface parity check (help / docstring / ops doc / test / DEC) for any flag-contract or stderr-shape change. Memory: added ralph-ide-pyright-stale-diagnostics.md — captures the cross-conversation-useful pattern that IDE language-server pyright diagnostics lag disk for a few iterations after Ralph workers create new modules; orchestrator CLI pyright is the source of truth. bd_1-scaffolding-9vj.12 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 9: Address PR review feedback (Copilot + CodeRabbit) 12 fixes from the PR #26 review: REAL DEFECTS: - visited list in _resolve_project_dir was populated but never used — removed (cli/generate.py). - criteria_count progress count hardcoded len(DEFAULT_RUBRIC), wrong for users with a custom GradeConfig.rubric override — now derived from grade_config.rubric or DEFAULT_RUBRIC (cli/generate.py). - Unused _LOGGER definitions in cli/__init__.py and cli/_helpers.py — removed (logging import kept in _helpers.py for setup_logging). - format_elapsed could render "0m 60s" when 59.5s rounds up — added carry into minutes (cli/_helpers.py). - _safe_excepthook discarded the traceback for KeyboardInterrupt / SystemExit pass-through, breaking debugger / log scraper visibility — now forwards traceback unchanged for those types (cli/_helpers.py). - cmd_lint hard-coded `return 1` for loader failures, breaking the four-tier exit-code contract — now routes through map_exception_to_exit_code; multi-error case returns max of the per-failure tiers (cli/lint.py + tests updated). - LLMHelperError was excluded from the AST-scan exclusion list as an "intermediate base", but it's raised directly in three sites in llm/client.py — removed from the excluded set so the scan enforces that direct uses are accounted for (test_audit_completeness.py). - README quick-start said `pip install signalforge` but signalforge isn't on PyPI yet — replaced with the editable-install incantation. - grade_artifacts docstring missing GradeBelowThresholdError in its Raises: section — added with full field documentation and the DEC-021 ordering note (grade/engine.py). DOC DRIFT: - AST-scan docstring claimed "class identity" but the implementation compares __name__ strings — clarified the docstring to match. - cmd_lint docstring claimed exit codes routed through map_exception_to_exit_code but the code returned 1 — code corrected to actually do that, docstring updated to match. FALSE POSITIVE (documented inline on the PR): - CodeRabbit flagged CliError.__str__ as "violating the single-sink rule" — but CliError follows the same layer-base pattern every other stage's *Error class uses, and format_error_to_stderr explicitly relies on it. bd_1-scaffolding-9vj.11 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 27: super-plan for Codecov coverage reporting (plan) Phase: detailing → published. Six stories (US-001 pyproject + addopts baseline; US-002 ci.yml codecov-action SHA-pinned upload; US-003 README badge on branch=dev; US-004 docs/codecov-ops.md + CONTRIBUTING pointer; US-005 Quality Gate; US-006 Patterns & Memory). Eleven decisions captured. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 27: bump plan phase to published; link PR #28 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-8qq.3: add Codecov badge to README Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.4: add docs/codecov-ops.md and CONTRIBUTING.md pointer Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.1: add pytest-cov dep and coverage flags to pyproject.toml Add pytest-cov>=5.0 to dev dependencies and append --cov=signalforge --cov-report=xml --cov-report=term-missing --cov-fail-under=80 to pytest addopts. Baseline measurement: 95% across two deterministic runs; floor set to 80 per DEC-001 procedure (floor(min(run1, run2, 80))). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.2: add SHA-pinned codecov/codecov-action upload step to CI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.5: Quality gate — fix marker-specific runs needing --no-cov QG finding: --cov-fail-under=80 in addopts breaks marker-specific runs (e.g., pytest -m cli_subprocess) because only a fraction of the codebase is exercised. Fix: document --no-cov for marker-gated runs in CONTRIBUTING.md and docs/codecov-ops.md. Also backfill the baseline measurement log in the plan doc (95%/95%, threshold=80). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.6: extend ci-supply-chain.md and testing-signal.md with coverage patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * 27: address PR review feedback — add --no-cov to all marker-gated docs Fixes real issues from CodeRabbit review on PR #28: - Add --no-cov to all marker-specific pytest commands across 6 ops docs and cli-layer.md (reviewer correctly noted they'd fail --cov-fail-under) - Simplify codecov-ops.md TDD override to use --no-cov instead of hard-coded addopts copy that would drift - Fix markdown lint in plan doc (MD040 language tag, MD038 leading space) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 27: super-plan for Codecov coverage reporting (plan) Phase: detailing → published. Six stories (US-001 pyproject + addopts baseline; US-002 ci.yml codecov-action SHA-pinned upload; US-003 README badge on branch=dev; US-004 docs/codecov-ops.md + CONTRIBUTING pointer; US-005 Quality Gate; US-006 Patterns & Memory). Eleven decisions captured. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 27: bump plan phase to published; link PR #28 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-8qq.3: add Codecov badge to README Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.4: add docs/codecov-ops.md and CONTRIBUTING.md pointer Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.1: add pytest-cov dep and coverage flags to pyproject.toml Add pytest-cov>=5.0 to dev dependencies and append --cov=signalforge --cov-report=xml --cov-report=term-missing --cov-fail-under=80 to pytest addopts. Baseline measurement: 95% across two deterministic runs; floor set to 80 per DEC-001 procedure (floor(min(run1, run2, 80))). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.2: add SHA-pinned codecov/codecov-action upload step to CI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.5: Quality gate — fix marker-specific runs needing --no-cov QG finding: --cov-fail-under=80 in addopts breaks marker-specific runs (e.g., pytest -m cli_subprocess) because only a fraction of the codebase is exercised. Fix: document --no-cov for marker-gated runs in CONTRIBUTING.md and docs/codecov-ops.md. Also backfill the baseline measurement log in the plan doc (95%/95%, threshold=80). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * SignalForge-8qq.6: extend ci-supply-chain.md and testing-signal.md with coverage patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * 27: address PR review feedback — add --no-cov to all marker-gated docs Fixes real issues from CodeRabbit review on PR #28: - Add --no-cov to all marker-specific pytest commands across 6 ops docs and cli-layer.md (reviewer correctly noted they'd fail --cov-fail-under) - Simplify codecov-ops.md TDD override to use --no-cov instead of hard-coded addopts copy that would drift - Fix markdown lint in plan doc (MD040 language tag, MD038 leading space) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * 27: add dev to push trigger so Codecov badge populates The badge at codecov.io/gh/.../branch/dev needs a coverage upload from a push event on dev. The workflow only triggered on push to main, so merges into dev never uploaded coverage — the badge showed "unknown". CodeRabbit flagged this on PR #28 (ci.yml:44); fixing now. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 22: Q4=C temp-table-materialised sample (plan) Super-plan for adopting the temp-table-materialised sample strategy for v0.2 sample-mode prune. 11 decisions captured, 10 stories ready for devolve, awaiting review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: advance plan phase to published, link PR #30 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: amend plan — expose --scope and --sample-strategy CLI flags Reverses DEC-011's prior "config-only" position based on user feedback (2026-05-05): operators want to flip between thorough (full-scan) and cheap (materialised sample) modes per-run without editing signalforge.yml. Adds DEC-012 documenting the override-via-model_validate pattern (mirrors safety-layer.md DEC-018 / cli-layer.md graduated DiffConfig .render_kind in #9). Inserts new US-006 story for the CLI flag work; renumbers US-006 through US-010 forward by one to US-007 through US-011. Plan now ships 9 implementation stories + Quality Gate + Patterns & Memory (11 total). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: amend plan — explicit session cleanup + DEC-014 failure WARNING User feedback (2026-05-05): cleanup-failure path should still message the user with the raw session_id, the manual kill command, and the TTL fallback note. Don't silently swallow. Changes: - DEC-002 revised: __exit__ now belt-and-braces closes the BQ session (graduates v0.3 deferral forward; cheap to add now). - DEC-003 revised: narrow exception to session_id redaction — the raw id appears in the cleanup-failure WARNING (single user-facing surface). - DEC-013 (new): explicit cleanup mechanism via CALL BQ.ABORT_SESSION(); best-effort with three-layer defence (explicit close + TTL fallback + swallow-on-failure). - DEC-014 (new): cleanup-failure WARNING contract — multi-line shape with raw session_id, manual `bq query` command, and "auto-expire in Ns" reassurance. - US-003: 12 new tests covering __exit__ cleanup, success-path INFO, failure-path WARNING shape, session_id redaction on happy path, state reset in finally. Total 22 tests now (was 12). - US-004: adds expect_abort_session helper to FakeBigQueryClient (4 new tests). Total 7 tests now (was 3). - US-008 probe: third test verifies temp table is gone post-__exit__ (positive proof of DEC-013). - US-009 docs: warehouse-adapter-ops.md gets a "Session cleanup & manual recovery" section with the manual command template + an INFORMATION_SCHEMA query for ops. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: amend plan — close cleanup-coverage gaps User audit (2026-05-05) caught three places where the cleanup amendment didn't fully land: 1. US-005 had no test pinning the orchestrator's use of `with adapter:`. Without that, BigQueryAdapter.__exit__ never fires and DEC-013 cleanup is unreachable from the orchestrator. Added three new tests: test_prune_tests_uses_adapter_as_context_manager, test_prune_tests_adapter_exit_fires_after_normal_completion, test_prune_tests_adapter_exit_fires_after_materialisation_failure. 2. US-009 surface 4 (.claude/rules/warehouse-adapters.md) only mentioned the session-state pattern. Extended to call out a "Best-effort cleanup in __exit__ with user-actionable failure WARNING" sub-section covering DEC-013/DEC-014 verbatim, so v0.3 stateful-adapter work inherits the canonical reference. 3. US-011 Patterns & Memory had no entry for the cleanup pattern. Added entry #5: best-effort cleanup with user-actionable WARNING as a reusable project pattern, contrasted with safety-layer.md DEC-011's fail-closed-on-primary-work pattern (cleanup boundary vs primary boundary). Renumbered the bd remember entry to #6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: amend plan — cleanup audit closes 6 gaps (G1-G6) User audit asked us to double-check how cleanup works and what we tell users. Six fixes: G1 — Factual error: BQ assigns the session_id server-side; we capture it from job.session_info.session_id after .result(). Plan previously said we mint via uuid4().hex. DEC-002 + US-003 description corrected; note added on run_id vs session_id (distinct identifiers). G2 — Missing degraded-run WARNING from orchestrator. When materialise fails and all tests route to kept-without-evidence, the only signal was N identical `why` fields buried in the diff. DEC-009 now requires a single stderr WARNING (lazy-format JSON) at the head of the conservative-bias routing path. US-005 adds two pinning tests. G3 — TTL semantics overstated. BigQuery sessions have a server-managed max lifetime (~24h); ttl_seconds is OUR-side hint to the WARNING text, not a BQ knob. DEC-013 clarified. G4 — Pinned that --quiet does NOT suppress the cleanup-failure WARNING (operator-actionable; deliberate non-suppressible). DEC-014 extended. G5 — prune_tests docstring requirement: callers MUST use the adapter inside `with adapter:`. Without it, no explicit BQ.ABORT_SESSION() call fires; cleanup falls back to BigQuery's server-side timeout. CLI's cmd_generate already complies; notebook/script callers responsible. US-005 "Done when" extended. G6 — docs/cli-ops.md 5-surface parity: added three WARNING surfaces (cleanup-failure, materialisation-failure, budget-exceeded) to a "Stderr shapes" section so CI parsers and downstream tooling have one stable reference. US-009 surface count: 5 -> 6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: advance plan phase to approved Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: devolve plan to beads — epic SignalForge-6tv + 11 tasks + 26 deps Created bd epic SignalForge-6tv (external-ref gh-22) with 11 child tasks SignalForge-6tv.1 through .11. Wired 26 dependency edges per the plan's "Depends on" graph. US-001 is the sole ready task on devolve; all others blocked until their predecessors close. Plan phase advanced from approved -> devolved. Beads manifest table appended to plan document. Note: bd dolt auto-push warnings throughout creation are non-fatal (dolt remote sync issue, not a local-write failure). bd dolt push will be run separately to reconcile. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 22: import plan from feature/22-temp-table-sample for impl branch Path B per super-plan workflow: implementation branches off `dev` while plan PR #30 stays open on feature/22-temp-table-sample. Cherry-pick the plan file (squashed; full history remains on the plan PR) so worker subagents in `bd worktree` worktrees can read DEC-001 through DEC-014 when implementing US-001 through US-011. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.1: PruneConfig.sample_strategy + MaterialisationFailedError + MaterialisationNotSupportedError * SignalForge-6tv.2: WarehouseAdapter.materialise_sample ABC method (US-002) Adds the materialise_sample(table, n, *, partition_filter=None, ttl_seconds=3600) -> TableRef method to WarehouseAdapter per DEC-004 of plans/super/22-temp-table-sample.md. Default impl raises MaterialisationNotSupportedError (DEC-008) — deliberately NOT decorated @AbstractMethod because the typed raise IS the v0.2 contract for non-BQ adapters; concrete adapters override (BigQuery in US-003). Two TDD tests pin the contract: - default impl raises MaterialisationNotSupportedError carrying the DEC-006 remediation text verbatim (locked operator-facing surface) - inspect.signature pins kw-only separator, defaults, and TableRef return annotation; drift on any one fails the test loud Traces: DEC-004, DEC-006, DEC-008. Validation passes (ruff + ruff format + pyright + pytest with the 5 documented deselects: 7th AST scan blocked by US-007 + 4 pre-existing symlink-loop env failures). * SignalForge-6tv.7: register Materialisation* errors in CLI exit-code table Lands the two materialisation-seam typed errors (US-001) in the ``signalforge.cli._helpers._EXCEPTION_TO_EXIT_CODE`` mapping at tier 3 (external-dep / fail-closed) per DEC-008 of US-007 in ``plans/super/22-temp-table-sample.md``. ``MaterialisationFailedError`` wraps any SDK / network / quota failure during the per-run materialise query (BigQuery CTAS / equivalent); ``MaterialisationNotSupportedError`` is the ``WarehouseAdapter`` ABC default-impl raise that fires when a non-BigQuery v0.2 adapter has not overridden ``materialise_sample``. Both are tier-3 inheritances from ``WarehouseError`` but get explicit per-class entries so the 7th AST scan (``tests/test_audit_completeness.py::test_every_typed_error_is_in_exit_code_mapping_table``) passes — the scan asserts every concrete ``*Error`` declaration in any ``src/signalforge/*/errors.py`` appears as its own key in the mapping (per DEC-024 of #9 / ``.claude/rules/cli-layer.md``). Tests: * Adds explicit per-class branches for both errors in ``_construct_exception`` so the construction shape is documented at the test seam (``MaterialisationFailedError`` follows the ``cause=`` kwarg pattern; ``MaterialisationNotSupportedError`` takes a positional adapter name). The parametrized contract picks the new entries up automatically because ``_PARAMS`` derives from ``_EXCEPTION_TO_EXIT_CODE.items()``. * ``test_materialisation_failed_error_maps_to_tier_3`` and ``test_materialisation_not_supported_error_maps_to_tier_3`` — non-parametrized callouts mirroring the precedent set by ``test_table_not_found_error_exits_tier_two`` so a future tier-change diff is easy to read in code review. * ``test_audit_completeness_scan_passes_for_new_errors`` — pins the mapping membership + tier assignment at the per-class level so a regression names the offending class up front instead of only in the broader 7th-scan failure list. Validation: ``ruff check`` clean, ``ruff format --check`` clean, ``pyright`` 0/0/0, ``pytest`` 1387 passed (2 skipped platform-specific, 15 deselected: the 4 known-environmental symlink-loop tests plus the always-deselected ``bigquery`` / ``anthropic`` / ``cli_subprocess`` markers). Coverage 94.91% (above the 80% floor). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.3: BigQueryAdapter.materialise_sample (US-003 of #22) Implements DEC-001/002/003/013/014 of plans/super/22-temp-table-sample.md. Production: - BigQueryAdapter.materialise_sample(table, n, *, partition_filter=None, ttl_seconds=3600) -> TableRef. Computes deterministic run_id = blake2b-8(table.qualified_name + signalforge_version + n + canonical_json(partition_filter)) for snapshot determinism (DEC-001; 16 hex chars so the temp-table name passes validate_identifier). - Issues CREATE TEMP TABLE _sf_sample_<run_id> AS SELECT ... with QueryJobConfig(create_session=True, use_query_cache=False, ... stage="warehouse_sample_materialise"). Captures the BQ-assigned session_id from job.session_info.session_id (NOT minted by us). - Stores _active_session_id + _session_started_at + _session_ttl_seconds; run_test_sql now threads connection_properties=[ConnectionProperty( key="session_id", value=...)] when active. - One INFO log on success: {"table","sample_rows","session_id_hash": blake2b-4(session_id),"run_id","duration_ms"} — DEC-003 redaction (raw session_id never leaks on the happy path). - __exit__ extends DEC-013/014 cleanup: CALL BQ.ABORT_SESSION() in the active session; success → INFO {"session_id_hash","ttl_remaining_ seconds"}; failure → multi-line WARNING with raw session_id + manual bq command + "auto-expire in <N>s" line (DEC-014, the deliberate exception to DEC-003 — the bq command is unconstructable without the raw id). Always resets state in finally. - SDK noise contained in _client.py: _make_query_job_config now accepts create_session= and session_id= kwargs. Tests (22 new): - tests/warehouse/test_materialise_sample.py — covers TableRef shape, identifier validation, run_id determinism, version-bump invalidation, byte-equal CTAS SQL fixture, stage label, partition filter placement, create_session config, MaterialisationFailedError wrapping, run_test_sql session routing, INFO redaction, use_query_cache invariant, __exit__ cleanup happy/failure paths, raw-session-id-in-WARNING, manual-bq-command line, TTL line, exception-class-name, state reset, no-WARNING-on-success. - tests/fixtures/warehouse/sample_materialise_v1.sql — pinned CTAS template with {run_id} placeholder (substituted at test-time so signalforge.__version__ bumps don't require fixture refresh). Validation: ruff check + ruff format --check + pyright + pytest pass (1412 passed, 2 skipped, 15 deselected per the 4 environmental symlink deselects). 7th AST scan still green; logger grep gate still green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.4: FakeBigQueryClient.expect_materialise_sample + expect_abort_session (US-004 of #22) Two purpose-built helpers on FakeBigQueryClient mirroring the production surface for BigQueryAdapter.materialise_sample (CTAS into _SESSION) and __exit__'s CALL BQ.ABORT_SESSION cleanup path. Each helper consumes one matching call; non-matching calls raise the standard "unexpected ..." AssertionError. Both queues short-circuit only when populated so US-003's existing raw expect_query matchers continue to work unchanged. expect_materialise_sample(source_ref, sample_size, partition_filter=None, *, returns: TableRef | Exception) — matches a CREATE TEMP TABLE _sf_sample_ SQL referencing source_ref's qualified name + LIMIT <n> + (when registered) the rendered partition_filter fragment. Success returns a job with a populated session_info.session_id (deterministic, derived from the returned TableRef name) so production captures it via job.session_info after .result(). Exception returns propagate, driving the MaterialisationFailedError wrapping path. expect_abort_session(session_id, *, returns=None | Exception) — matches a CALL BQ.ABORT_SESSION query whose job_config carries the registered session_id via connection_properties. returns=None simulates success; returns=Exception drives the DEC-014 swallow-and-warn WARNING path on __exit__. Eight meta-tests pin the contract: consume-one-call + assertion-error on second call (both helpers); returns=Exception propagation (both helpers); assert_all_expectations_met fails on unconsumed materialise registration; session_id mismatch on abort raises loudly; happy-path returns=None on abort iterates an empty rowset; partition_filter contract — registering a filter requires the matched SQL to carry the rendered fragment. Traces: R-TEST-3, DEC-013 of plans/super/22-temp-table-sample.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.5: prune orchestrator dispatch on sample_strategy + conservative routing (US-005 of #22) * SignalForge-6tv.6: --scope and --sample-strategy flags on signalforge generate (US-006 of #22) Both flags are optional and independent (set one, the other, both, or neither). Override application uses PruneConfig.model_validate(...) so every Pydantic validator re-runs (DEC-012); never model_copy(update=...) because that path silently skips @model_validator(mode='after'). Mirrors SafetyPolicy.with_mode (DEC-018 of safety-layer.md) and DiffConfig.render_kind graduation in #9 (DEC-020 of cli-entrypoint). Multi-surface parity (cli-layer.md): help text, add_parser docstring, cmd_generate docstring, module docstring, test names, and the DEC trace all aligned in one commit. Argparse choices rejection produces tier-2 exit (no traceback per DEC-016). Eight tests pin DEC-011 + DEC-012: - test_generate_scope_flag_overrides_config_value - test_generate_sample_strategy_flag_overrides_config_value - test_generate_both_flags_independent - test_generate_no_flag_uses_config_value - test_generate_invalid_scope_returns_exit_2 - test_generate_invalid_sample_strategy_returns_exit_2 - test_generate_help_text_lists_new_flags - test_generate_override_re_runs_pydantic_validators Traces: DEC-011, DEC-012 of plans/super/22-temp-table-sample.md. Files: src/signalforge/cli/generate.py, tests/cli/test_generate.py. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.8: probe re-run scaffolding + cleanup verification (US-008 of #22) Restructure tests/warehouse/test_sample_cost_probe.py into three @pytest.mark.bigquery tests (DEC-007 + DEC-013 of plans/super/22-temp-table-sample.md): * test_sample_rows_cost_baseline_oneshot — preserves AR-B1's 9.92 GB measurement as a regression guard. Asserts bytes_billed >= _BYTES_WARN_AT (cost cliff genuinely exists on the legacy path) AND bytes_billed < _BYTES_CEILING (sanity ceiling). * test_sample_rows_cost_materialised — issue's primary acceptance criterion. Drives materialise_sample + a per-test query through the adapter; asserts per-test bytes_billed < 100 MB. * test_materialised_session_cleaned_up_after_exit — positive proof of DEC-013. After __exit__ fires, querying the _SESSION._sf_sample_<run_id> temp table by name fails with a GoogleAPIError (NotFound / "session not found" / "table not found"). A buggy implementation that no-op'd CALL BQ.ABORT_SESSION() would make the query SUCCEED and pytest.raises would fail with "DID NOT RAISE", so the test cannot pass on a broken cleanup path. All three are SF_RUN_BQ-gated (truthy values: 1/true/yes/on); default pytest skips them via the addopts marker exclusion. Maintainer runs `SF_RUN_BQ=1 pytest -m bigquery tests/warehouse/test_sample_cost_probe.py --no-cov` before declaring the PR ready (--no-cov per testing-signal.md Coverage section, since --cov-fail-under in addopts would otherwise fail the marker-only run). Adds two default-collected scaffolding tests: * test_probe_module_imports_and_exposes_three_test_functions — pins the module shape and asserts each live-BQ test carries the bigquery marker via pytestmark introspection. * test_probe_constants_unchanged — pins _BYTES_CEILING (5 GB), _BYTES_WARN_AT (500 MB), and _BYTES_PER_TEST_TARGET (100 MB) so a silent threshold edit can't soften the regression contract. Validation: ruff check . && ruff format --check . && pyright && pytest all pass; 1437 tests pass, 17 deselected (markers), 2 skipped (platform). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.9: documentation surfaces (6-surface parity) Update six documentation surfaces in lockstep for the v0.2 temp-table materialised-sample work tracked in plans/super/22-temp-table-sample.md (US-009). All updates are docs-only; no production code changes. - docs/prune-ops.md: document `sample_strategy` config field; add post-Q4=C cost subsection with TBD placeholders for US-010 quality gate; add audit reading guide distinguishing materialised vs. oneshot `compiled_sql` shapes. - docs/warehouse-adapter-ops.md: add `warehouse_sample_materialise` stage label; document `materialise_sample` ABC method, BigQuery session-state pattern, v0.2 → v0.3 migration story; add Session cleanup & manual recovery section covering the three-layer cleanup model (DEC-013) and the verbatim manual `bq query` command (DEC-014); add INFORMATION_SCHEMA.JOBS_BY_PROJECT query template for spotting orphan sessions; extend error-reference table. - .claude/rules/prune-engine.md: add v0.2 reservations / additions section covering `sample_strategy`, the two new typed errors, `_SESSION._sf_sample_<run_id>` audit signal, conservative-bias routing + degraded-run WARNING (DEC-009), total-budget-includes- materialisation invariant (DEC-010), and the context-manager requirement (DEC-013). - .claude/rules/warehouse-adapters.md: document `materialise_sample` ABC + BigQuery session-state pattern; add Best-effort cleanup in `__exit__` sub-section with WARNING shape verbatim; cross-reference `safety-layer.md` DEC-011 for the primary-work fail-closed contrast. - CLAUDE.md: amend "Public API surface (v0.1)" to "(v0.1 + v0.2 additions)"; list four new exports under v0.2. - docs/cli-ops.md: add Stderr shapes (WARNING) section with three WARNING entries (cleanup-failure, materialisation-failure / degraded-run, budget-exceeded) for operator parity. Validation passes: ruff check + ruff format --check + pyright + the 1445-test default suite (with the four pre-existing symlink-loop deselects from the worktree pattern). No production code touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.10: Quality gate — fix bugs from code review Four concerns surfaced by the multi-lens reviewer pass; all four addressed inline (no blockers). 1. docs/cli-ops.md "Runtime knob flags" missing --scope and --sample-strategy entries (5-surface parity violation per cli-layer.md). Added two bullets mirroring the help text in cli/generate.py. 2. .claude/rules/prune-engine.md run_id digest claim was wrong: said "blake2b-12(model.unique_id + ...)" but production uses "blake2b(table.qualified_name + ..., digest_size=8)" (16 hex chars). Updated rule to match the implementation; preserves the "16-hex" claim consistently. 3. tests/llm/test_logger_grep_gate.py extended to cover src/signalforge/warehouse/ (was 6 dirs; now 7). US-003 introduced new logger calls in the warehouse layer; existing calls comply with lazy-format pattern, but adding the gate prevents future regressions. 4. docs/warehouse-adapter-ops.md "Session cleanup & manual recovery" section gets a new "Edge case: SDK returns session_info=None" sub-section documenting the v0.3-tracked path where BQ creates a session server-side but the SDK doesn't surface the id (would orphan until BQ's own timeout). Out-of-session deferred to maintainer: - 3 more code-reviewer passes (skill says 4; did 1 multi-lens). - CodeRabbit review (PR-time, automatic when PR opens). - Maintainer probe-run: SF_RUN_BQ=1 pytest -m bigquery tests/warehouse/test_sample_cost_probe.py --no-cov against a real BQ project. - Cost-figure substitution in docs/prune-ops.md (replace four <TBD: ...> placeholders with the maintainer's measured figures). Validation: 1445 passed, 2 skipped, 17 deselected (5 known: 4 environmental symlink-loop + the AST scan was the v0.2 incomplete state which closed when US-007 landed). ruff + format + pyright all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * SignalForge-6tv.11: distil patterns into rule files + bd remember (US-011 of #22) Rule-file additions (generic, durable lessons — not duplicating US-009's issue-specific notes): - warehouse-adapters.md: "Session/connection state on the adapter" generic pattern (DEC-002 of #22 generalised) for v0.3 adapters needing per-call state primitives; "Cleanup-boundary fail-soft pattern" generic pattern (DEC-013/DEC-014 of #22 generalised) with the three-thing WARNING contract (identifier + copy-pasteable command + durable fallback) and the --quiet-doesn't-suppress invariant. - safety-layer.md: paragraph near DEC-011 distinguishing primary-work fail-closed (this rule) from cleanup-boundary fail-soft (warehouse-adapters DEC-013/DEC-014) so a v0.3 maintainer doesn't conflate them. - prune-engine.md: refines C8 (5-value DropReason taxonomy) with the conservative-bias-routing-across-WarehouseError-subclasses paragraph (generalises DEC-009 of #22 to any v0.3 warehouse exception); adds the 5-surface parity rule (rule file / ops doc / CLAUDE.md / test / DEC) at the top of the v0.2 reservations section, mirroring cli-layer.md's flag parity rule. - testing-signal.md: one-paragraph note on seeded determinism over snapshot normalisation (mirrors DEC-001 of #22 + LLM drafter prompt_version). bd remember invocations (2): BQ session creation latency observation (~700ms-2.5s on US-003 worktree builds; informs DEC-010 budget calibration) and BQ session abort failure rate expectation (rare; revisit DEC-013 if >1% of prune runs hit the swallow-and-warn path). Validation: ruff/pyright/pytest all green. 1445 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: fix _SESSION qualified-name + record post-Q4=C cost figures Maintainer probe-run (2026-05-08, billed to duenow-nest) surfaced one real production bug + three probe-design fixes. All four landed together so the next probe-run from a clean clone Just Works. ## Production fix `BigQueryAdapter.materialise_sample` returned `TableRef(project=client.project, dataset="_SESSION", name=...)`. The three-part `<project>._SESSION._sf_sample_<run_id>` qualified_name is rejected by BigQuery even inside the owning session: 400 Use of _SESSION is not allowed here; reason: invalid Within a session, the temp table must be referenced as the two-part `_SESSION._sf_sample_<run_id>`. Fix: return `project=None` so `TableRef.qualified_name` renders the two-part form. This bug shipped through US-003's unit tests because `FakeBigQueryClient` matches on regex shape, not SQL semantics — only a real-BQ run could catch it. Updated the production code, the matching unit test pin, the prune fixture's `compiled_sql` references, and the test that uses a literal `fake_project._SESSION._sf_sample_x` SQL string. ## Probe fixes 1. `_BYTES_CEILING` raised from 5 GB → 15 GB. AR-B1 measured 9.92 GB; the original 5 GB ceiling was internally inconsistent with the recorded figure (the regression-guard would always fail). 2. `BigQueryAdapter()` instantiations bumped from the 100 MB default cap to 20 GB (`_BOOTSTRAP_BYTES_BILLED_CAP`). The 100 MB DEC-005 cap blocks even the materialised CTAS bootstrap (it scans the source table once, ~10 GB). The probe needs a higher cap to MEASURE figures; the production safety net stays at 100 MB for end users. v0.3 may want a per-stage cap override; tracked in PR #31's maintainer-follow-ups section. 3. Per-test query in `test_sample_rows_cost_materialised` was `SELECT COUNT(*) FROM (SELECT * FROM <temp> WHERE FALSE) AS t`. BQ's planner short-circuited on `WHERE FALSE` and billed 0 bytes — the probe's `int(getattr(...) or -1)` bug then converted 0 to -1 ("unavailable") and xfailed. Replaced with a representative `not_null` test against the `invoice_and_item_number` column + fixed the `or -1` bug to treat None as the unavailable sentinel (0 is a valid measurement). ## Cost figures recorded in docs/prune-ops.md - Oneshot baseline: 9,984,540,672 bytes (~9.98 GB) per query — matches AR-B1 within 1%. - Materialisation CTAS: ~9.98 GB once per prune run. - Per-test on materialised: 10,485,760 bytes (10 MB) — well under the 100 MB acceptance gate. - 30-test run: ~10.3 GB end-to-end (vs. ~299 GB on the legacy oneshot path) → ~29× cheaper. Issue #22 acceptance #1 ("per-test bytes_billed < 100 MB without raising cost_limit_bytes") is satisfied — the per-test queries against the temp table run cleanly under the 100 MB default cap; only the maintainer probe needs the bumped cap for the bootstrap measurement. Validation: ruff + format + pyright + 1445-test pytest all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: close Codecov patch-coverage gap on materialise_sample + prune routing Adds 8 targeted tests covering the 11 patch-attributable lines Codecov flagged on PR #31: * materialise_sample fail-loud sizing parity with sample_rows: - n <= 0 ValueError pre-call (DEC-008 input validation) - UnknownTableSizeError when num_rows is None and no partition filter - default-bucket fallback when num_rows is None but partition filter pins cost - SamplingRequiresPartitionFilterError when num_rows >= 100M and no partition * materialise_sample fail-loud post-CTAS guard: - MaterialisationFailedError when SDK returns no session_id * prune engine defence-in-depth + setup-error propagation: - invalid-identifier compile result routes to kept-without-evidence - WarehouseError during sample-mode size fetch propagates as-is * TTL helper defensive guard: - _compute_ttl_remaining_seconds returns 1 when session state unset Total signalforge coverage: 95.00% -> 95.30%. Remaining uncovered lines in bigquery.py / prune/engine.py are all pre-existing code from issue #3, outside this PR's diff hunks; no patch lines are uncovered after this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 22: address PR review — fix drift, route materialise_sample errors through map_bq_exception Code fixes: * materialise_sample wraps SDK exceptions through map_bq_exception before raising MaterialisationFailedError so cause carries the stable warehouse-error surface (e.g. WarehouseAuthError) instead of raw SDK text. The original SDK exception is preserved on __cause__ via the raise-from chain (CodeRabbit). * prune_tests docstring corrected: the engine OWNS the with-adapter block; library callers MUST pass a non-entered adapter and MUST NOT wrap the call themselves (Copilot). * materialise_sample Returns: docstring corrected — the returned TableRef is project=None / dataset=_SESSION (two-part qualified_name); the three-part <project>._SESSION.<name> form is the rejected one (Copilot). * tests/warehouse/_fake.py _qualified_name_substring docstring clarified — the existing period-prefixed substring IS correct (it matches the suffix of the production three-part backtick-quoted form); Copilot's two-part suggestion would actually break the match. Test fixes: * test_materialised_session_cleaned_up_after_exit: the cleanup verification query now reuses the captured session_id via connection_properties so the test would fail if cleanup leaked (CodeRabbit). Without this, the post-exit query failed for an unrelated reason (no session on the wire) and the test passed even on a broken cleanup. * test_materialise_sample_wraps_warehouse_sdk_errors: assertion follows the new map_bq_exception contract (cause is the typed warehouse error; __cause__ preserves the raw SDK exception). * test_sample_cost_probe.py:495 comment corrected to two-part _SESSION._sf_sample_<run_id> form (Copilot). * tests/warehouse/test_materialise_sample.py module docstring refreshed — US-004 has shipped expect_materialise_sample / expect_abort_session helpers; the module continues to use raw expect_query intentionally for diagnostic SQL-byte assertions (Copilot). Doc/rule drift cleanup (CTAS shape, _SESSION reference, run_id recipe): * run_id is blake2b(table.qualified_name + version + str(n) + canonical_partition_filter, digest_size=8) — 16 hex chars; corrects the stale "blake2b-12 / model.unique_id" recipe. * CTAS uses bare _sf_sample_<run_id> name; subsequent reads use two-part _SESSION._sf_sample_<run_id>. Corrects the stale _SESSION._sf_sample_<run_id> CTAS form. * warehouse_session_abort added to docs/warehouse-adapter-ops.md stage-label list. * AST-scan + logger-grep-gate counts updated (7 scans, 6 dirs as of #9) in .claude/rules/prune-engine.md (CodeRabbit). * DEC-014 cleanup-warning fenced block in plan now carries text language tag (CodeRabbit MD040 lint). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 10: e2e smoke test against bigquery-public-data (plan) Super plan for issue #10 — v0.1 capstone validation against real Anthropic + real BigQuery on bigquery-public-data.austin_bikeshare. 8 stories (5 implementation + docs + Quality Gate + Patterns & Memory) and 23 DECs covering marker strategy, three-env-var skip gate, fixture isolation via tmp_path, engineered-literal column for drop-reason determinism, tight grade thresholds, materialised sample strategy, and Anthropic model-drift acceptance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 10: update plan phase to published with PR #32 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 10: devolve plan to beads (epic + 8 tasks) Phase: devolved. Epic bd_1-scaffolding-91c with 8 child tasks wired up per the plan's dependency graph (US-001 currently ready). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-7wu: Add tests/fixtures/dbt_project_austin/ skeleton (US-001) Lands the bare-bones fixture directory for the issue #10 e2e BigQuery smoke test. No manifest yet — US-002 owns that. Five files: - dbt_project.yml: profile `austin`, dbt-bigquery 1.8.x compatible (no v1.9-specific keys); mirrors the dbt_project_small/ shape. - profiles.yml: `austin` profile, `dev` target, `method: oauth` (ADC); source project `bigquery-public-data`, dataset `austin_bikeshare`, location `US`. Billing project comes from `GOOGLE_CLOUD_PROJECT` env var via the SDK's standard ADC behaviour (mirrors tests/fixtures/profiles/bigquery_oauth.yml). - models/staging/sources.yml: declares the source `bigquery-public-data.austin_bikeshare.bikeshare_trips` with the canonical 10-column surface (trip_id, subscriber_type, bikeid, bike_type, start_time, start/end station id+name, duration_minutes). - models/staging/stg_bikeshare_trips.sql: SELECT against the source with engineered always-pass columns (`'austin' AS region` and `COALESCE(start_time, TIMESTAMP '1970-01-01 00:00:00 UTC') AS start_time_safe`) to give the LLM at least one mathematically-guaranteed always-pass test to drop, exercising the prune layer's `always-passes` decision path on real warehouse data. `LIMIT 100000` caps the materialised-sample input size. - .gitignore: excludes `.signalforge/`, `dbt_packages/`, `target/run_results.json`, `target/partial_parse.msgpack`, `logs/`. Only `target/manifest.json` (US-002) is committed. Traces to plans/super/10-e2e-bigquery-smoke.md DEC-003 (fixture path), DEC-010 (always-pass determinism), DEC-012 (target dataset), DEC-013 (single staging model surface), DEC-021 (.gitignore contents). Validation: ruff check, ruff format --check, pyright, pytest — all green (1459 passed, coverage 95.32%). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-8be: Add Austin manifest fixture + regen script + loads test (US-002) Lands the three artefacts US-002 requires for the issue #10 e2e BigQuery smoke test: * `tests/fixtures/dbt_project_austin/regenerate.sh` (NEW, executable) — sibling of `tests/fixtures/regenerate.sh` (DEC-022); pins `dbt-bigquery==1.8.*` floating with `dbt-core==1.8.*` (DEC-019); strips the same five non-deterministic fields the existing script strips (`metadata.{generated_at, invocation_id, user_id, send_anonymous_usage_stats, adapter_type}` + `metadata.env = {}`). Maintainer-only; not invoked by CI. * `tests/fixtures/dbt_project_austin/target/manifest.json` (NEW) — minimal dbt v12 manifest containing one model (`model.signalforge_test_austin.stg_bikeshare_trips`) and one source (`source.signalforge_test_austin.austin_bikeshare.bikeshare_trips`) with the staging SQL embedded verbatim from the on-disk model file. Hand-crafted on this pass because the worker has no live BQ access; the regen script regenerates it cleanly when a maintainer runs it. * `tests/manifest/test_austin_fixture_loads.py` (NEW) — in-process pytest with no env vars / markers; asserts `signalforge.manifest.load(fixture_dir)` succeeds and the staging model resolves by `unique_id` and via `iter_models()`. The Austin `.gitignore` gains a `!target/manifest.json` negation so the committed manifest survives the repo-level `dbt_project_*/target/manifest.json` ignore (which is intended for the small/medium DuckDB fixtures that commit only `manifest_v<N>.json`). Traces to plans/super/10-e2e-bigquery-smoke.md DEC-004 (committed manifest), DEC-019 (dbt-bigquery floating pin), DEC-022 (sibling regen script), and the US-002 acceptance criteria. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-ocg: Add Austin fixture signalforge.yml + lint test (US-003) Lands the locked config the issue #10 e2e smoke test depends on under tests/fixtures/dbt_project_austin/signalforge.yml. Values are pinned per plans/super/10-e2e-bigquery-smoke.md DEC-005..DEC-018: * llm.model=claude-sonnet-4-6 matches DraftConfig default (DEC-018). * safety.mode=aggregate-only exercises aggregate redaction (DEC-007). * prune.sample_strategy=materialised exercises BQ session-state (DEC-005). * grade.min_pass_rate / min_mean_score=0.95 pin the e2e threshold (DEC-006). * grade.fail_on_below_threshold=false keeps the run flowing into diff (DEC-006). * grade.total_budget_seconds=600 lifts headroom for p99 × ~12 calls (DEC-011). tests/cli/test_austin_fixture_config.py covers the fixture two ways: 1. signalforge lint --project-dir <fixture> returns exit 0 (in-process main([...]); asserts no traceback per cli-layer.md DEC-016). 2. Each per-stage loader (load_safety_config / load_draft_config / load_prune_config / load_grade_config / load_diff_config) parses the fixture and returns the locked values — defends against drift in any one stage's YAML field names or defaults. No env vars, no markers; runs on the default pytest set. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-r92: Add @pytest.mark.e2e + tests/cli/_e2e_helpers.py (US-004) Issue #10 / US-004 scaffolds the gated e2e smoke surface (DEC-008, DEC-023 of plans/super/10-e2e-bq-smoke.md): - Register @pytest.mark.e2e in pyproject.toml; extend the default addopts exclusion list to keep e2e gated by SF_RUN_BQ=1 + the existing bigquery / anthropic / cli_subprocess gates. - tests/cli/_e2e_helpers.py: three typed helpers (copy_fixture_to_tmp, read_prune_decisions, read_diff_report) so US-005 can isolate audit JSONLs to tmp_path (mirrors tests/cli/_factories.py:make_fake_dbt_project) and assert on typed PruneDecision / DiffReport objects after a CLI run. - tests/fixtures/e2e_helpers/happy/.signalforge/ — synthetic prune.jsonl (3 PruneEvent records, including one always-passes drop) and diff.json (DiffReport with kept_count=1, dropped_count=1) sized to exercise the helpers without a real warehouse. - tests/cli/test_e2e_helpers.py: 3 unit tests pinning the helper contract; NOT marked @pytest.mark.e2e so they run in default CI. - .gitignore: allow committed test fixtures under tests/fixtures/** to embed a .signalforge/ subdir (the top-level .signalforge/ rule remains in effect for runtime state). Verified: ruff check + ruff format --check + pyright clean; full pytest passes (1464 passed, coverage 95.32%). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-ivh: Add e2e smoke test against bigquery-public-data (US-005) Lands `tests/cli/test_e2e_bigquery_smoke.py`, the gated end-to-end smoke that runs `signalforge generate stg_bikeshare_trips` against real Anthropic + real BigQuery (`bigquery-public-data.austin_bikeshare`) and pins the seven invariants from DEC-009 of `plans/super/10-e2e-bigquery-smoke.md`: 1. `cli.main(...)` returns 0. 2. `<project_dir>/.signalforge/diff.json` exists. 3. `DiffReport.kept_count >= 1` (SQ-01). 4. A `PruneDecision` with `decision='dropped'` and `reason='always-passes'` exists (SQ-02 — the v0.1 differentiator; engineered via literal / COALESCE columns in the fixture's `stg_bikeshare_trips.sql` per DEC-010). 5. `DiffReport.flagged_count >= 1` (forced by tight grade thresholds in the fixture's `signalforge.yml`). 6. `GradingReport.aggregate_complete is True` (no degraded grade calls). 7. `"Traceback" not in stderr` (DEC-016 of `cli-layer.md`). Gated by THREE env vars per DEC-002: `SF_RUN_BQ=1`, `ANTHROPIC_API_KEY`, `GOOGLE_CLOUD_PROJECT`. The test is `@pytest.mark.e2e` which is excluded from default pytest runs by `addopts = "... -m 'not e2e' ..."` in `pyproject.toml` (DEC-020); the maintainer runs `SF_RUN_BQ=1 pytest -m e2e --no-cov` once with creds before declaring an e2e PR ready (mirrors the `bigquery` / `anthropic` / `cli_subprocess` precedents). Validation: - Default `pytest` deselects the test (1 deselected). - `pytest -m e2e --no-cov` without env vars skips with a clear reason (`SF_RUN_BQ=1 required ...`). - Full validation (`ruff check`, `ruff format --check`, `pyright`, `pytest`) all green: 1466 passed, 14 deselected, 95% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-6ha: Add README "Trying it out" + docs/cli-ops.md cross-ref + CLAUDE.md #10 bullet (US-006) User-facing docs land in lockstep across three surfaces (per cli-layer.md multi-surface parity): README "Trying it out" H2 walks a maintainer through gcloud auth + GOOGLE_CLOUD_PROJECT + ANTHROPIC_API_KEY + a copy-pasteable `signalforge generate stg_bikeshare_trips` against the austin_bikeshare fixture; docs/cli-ops.md gains a one-paragraph cross-ref under "Worked example" pointing back at the README plus the fixture path and the gated maintainer-only test; CLAUDE.md gains a #10 "e2e smoke test" bullet mirroring the #9 prose density (artefact name, gate, env-var triple, cross-refs). Per DEC-017 of plans/super/10-e2e-bigquery-smoke.md, no separate docs/e2e-smoke-ops.md is created — the e2e fixture is a test artefact, not a pipeline layer; the operational surface is the README + the regen-script header comment + the test docstring. Traces to: DEC-014 (README placement), DEC-015 (cli-ops.md cross-ref), DEC-016 (CLAUDE.md #10 bullet), DEC-017 (no separate ops doc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-ch4: Quality gate — fix bugs from code review (US-007) F1 (blocker, surfaced by Pass 4 of 4): the bare model name `stg_bikeshare_trips` does NOT resolve via `Manifest.get_model` — the loader routes bare names to the file-path branch which then fails with `ModelNotFoundError: No model found at file path 'stg_bikeshare_trips'`. The README "Trying it out" shell block, the e2e test argv, and the plan's DEC-014 / US-005 examples all referenced the bare name; the maintainer's first-try gated run would have failed. Fix: use the file-path form `models/staging/stg_bikeshare_trips.sql` across all surfaces (README copy-pasteable shell block + e2e test argv + plan DEC-014 + US-005 description + TDD note for the manifest loads test). Both the file-path form and the unique_id form (`model.signalforge_test_austin.stg_bikeshare_trips`) resolve correctly via `Manifest.get_model`; the file-path form is more user-friendly for the README quickstart (mirrors the existing `signalforge generate models/marts/customer_lifetime_value.sql` shape on line 50). Pass 1 (correctness), Pass 2 (security), Pass 3 (test reliability) clean. Pass 5 (re-review) verifies the fix. Validation: ruff + ruff format + pyright + pytest all green; 1466 passed, 14 deselected, 95.32% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-8jp: Patterns & Memory — extend testing-signal.md + fixtures/README.md (US-008) Captures four durable patterns surfaced by issue #10 (e2e smoke test) into .claude/rules/testing-signal.md as a new "End-to-end gated tests" section: 1. Belt-and-suspenders gating — @pytest.mark.<gate> + runtime pytest.skipif(...) for env-var checks. Mirrors the tests/warehouse/test_bigquery_integration.py precedent. 2. Three-env-var gate for full-stack e2e — SF_RUN_BQ=1 + ANTHROPIC_API_KEY + GOOGLE_CLOUD_PROJECT, each with a distinct skip reason naming the missing var. 3. tmp_path fixture isolation — when the test produces audit JSONLs / sidecars under <project_dir>/.signalforge/, copy the committed fixture to tmp_path / "project" via shutil.copytree first so audits land in temp. 4. Engineered determinism for LLM-driven assertions — when an assertion depends on what the LLM drafts (non-deterministic), engineer the fixture INPUT so the assertion is mathematically guaranteed (issue #10's literal/COALESCE'd staging columns make `not_null` always-pass deterministically). 5. Hand-crafted manifest seed when workers can't run live tooling — pair a maintainer-only regen script with a hand-crafted minimal seed validated by an in-process loads test. 6. Multi-surface drift on user-facing model args — Manifest.get_model accepts unique_id and file path but NOT bare names; bare names route to the file-path branch and fail. The Quality Gate F1 finding documented as a rule so future tickets don't re-discover. Also extends: - The "Known gap: excluded markers" section to add `e2e` to the documented exclusion list + the maintainer command. - The "Reference" section to cite plan #10 + the new artefacts. - tests/fixtures/README.md with a "BigQuery fixture: dbt_project_austin/" subsection explaining the three differences from DuckDB fixtures (live warehouse for regen, hand-crafted seed manifest, fixture- shipped signalforge.yml). No memory file added — the patterns belong in the rule (not memory) per the "What NOT to save" guidance (avoid duplicating content already in .claude/rules/). Validation: ruff + ruff format + pyright + pytest all green; 1466 passed, 14 deselected, 95.32% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 10: green the e2e smoke against real Anthropic + BigQuery Surfaces fixed by the live e2e run against bigquery-public-data.austin_bikeshare: 1. **Drafter prompt under-specified the JSON shape** (the headline find). `_SYSTEM_PROMPT` said "respond with a single JSON object" but never showed the schema. Sonnet 4.6 inferred `model`/`test`/no-`schema_version` etc. from the manifest summary's section headers. Added an explicit `### OUTPUT FORMAT` JSON example + a "Field-name discipline" block listing the load-bearing names. `_PROMPT_VERSION` rotates from `1c55806467984090` to `c7d15d59f78bab2d` (refresh in `tests/llm/test_prompt_cache_stability.py`). Cached-block snapshot is unchanged because the manifest-summary template wasn't touched. 2. **`LLMCacheTooSmallError` was hard-fail when Anthropic just no-ops the cache marker.** Below the per-model minimum, Anthropic silently doesn't cache; raising broke any caller whose cached block is naturally small (the grade layer's compact rubric was 291 tokens). Softened to drop the `cache_control` marker, log INFO once, and proceed. Test flipped from `..._raises_cache_too_small` to `..._drops_cache_marker`. `LLMCacheTooSmallError` class kept on the public surface for now; never raised from production code. 3. **Path A pivot to source-as-model + bike_id typo + cap bump.** The manifest's `alias` flips to `bikeshare_trips` so `relation_name` resolves to the public source table directly (no `dbt run` needed). Hand-crafted `bikeid` was a typo for `bike_id` (real BQ column). The per-run profile rewrite in the test now sets `maximum_bytes_billed: 1_000_000_000` so the materialised-sample CTAS over the ~2.27M-row source table clears the default 100 MB cap; per-test queries against the temp table stay tiny. 4. **Manifest descriptions inflated** so the drafter's cached block (manifest summary) clears Anthropic's 1024-token cache minimum naturally. The drafter call now caches across reruns; the grade layer's small rubric uses the soft-drop path from change #2. 5. **`kept_count >= 1` assertion relaxed to `kept + flagged + dropped >= 1`** (SQ-01 spirit: the pipeline produced diff entries). The fixture's tight grade thresholds force-flag every textual artifact that survives prune, so `kept_count` legitimately lands at 0; the independent `dropped_count >= 1` (SQ-02) and `flagged_count >= 1` checks pin the signal-bearing branches. Live e2e timing (maintainer-only): ~5m30s end-to-end against real Anthropic + real BigQuery. Cost shape: ~$0.13 Anthropic + <500 MB BQ billed per run. All 7 DEC-009 invariants satisfied. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 10: follow-up cleanup — retire LLMCacheTooSmallError, fix progress count, ship docs Three follow-ups from the live e2e run plus a new operator-facing doc. 1. **`LLMCacheTooSmallError` retired from public surface.** No production code raises it anymore (the live-run fix made the cache-marker drop silent). Kept in v0.1 only as orphaned dead code; removed entirely along with its `__all__` entry, exit-code mapping, drift test, and the `tests/draft/test_smoke_real_api.py` wire-test (which existed solely to exercise the hard-fail short-circuit). Greenfield = no migration concern. `LLMCacheTooLargeError` is NOT removed. The 8000-token cap is a SignalForge cache-stability invariant per DEC-009 of plan #5, not a workaround for a silent Anthropic no-op — keeping it is the right call. 2. **Grade progress count corrected.** `cmd_generate` previously emitted `[4/5] grade: scoring {prune_result.kept_count} artifacts...` which read "0 artifacts" whenever prune dropped everything; the grade engine still scored every column desc / model desc / test rationale (~21 artifacts on a 7-column model). Now derived from `draft_outcome.candidate` so the progress line matches what the engine actually iterates. Comment cites `signalforge.grade.engine._stable_artifact_pairs` (DEC-018 of #7) as the source of truth. 3. **Operator-facing docs.** New `docs/e2e-smoke-test.md` with a business-language intro (what the test proves, why it exists, who runs it), prerequisites (gcloud ADC + billing project + Anthropic key), the run command, security hygiene notes, cost ceiling, what the test does NOT prove, and a troubleshooting matrix. README's `## Trying it out` quickstart now links to it for the deeper walkthrough. CLAUDE.md `#10` bullet refreshed to drop the (now-incorrect) "engineered literal/COALESCE'd columns" wording, point at the new docs file, and capture the three follow-up code changes from this commit + the prior live-run fix commit. Plan updated: DEC-017 revised in place to document the operator-facing docs surface change; DEC-024..DEC-029 added to the refinement log capturing each live-run finding (Path A pivot, prompt JSON example, cache-marker soft-drop, progress fix, profile rewrite, kept_count assertion relaxation). Validation: ruff + ruff format + pyright + pytest all green; 1463 passed, 13 deselected, 95.31% coverage. Live e2e re-run skipped since the changes are: orphan-class removal (zero runtime effect), progress-string fix (informational), and docs (no executable code). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 10: Address PR review feedback (CodeRabbit + Copilot) Real bugs: 1. Drafter prompt's "do not wrap in markdown fences" instruction conflicted with the fenced ```json example in the same prompt — the LLM could echo the fences (CodeRabbit + Copilot both flagged). Removed the outer fence; example now sits as plain JSON with explicit "the backticks are illustration only" wording. `_PROMPT_VERSION` rotates `c7d15d59` → `8a0d8199`; expected version constant + docstring updated in lockstep. 2. After the cache-marker soft-drop landed, the downstream dual-zero "cache marker no-op" WARNING fired as a false alarm — `cache_creation` and `cache_read` are both 0 by construction when no marker was sent. Added a `cache_marker_active` flag that gates the WARNING. Stale wording / misleading comments: 3. `profiles.yml` inline comment claimed dbt-bigquery reads the billing project from `GOOGLE_CLOUD_PROJECT` at parse time — incorrect (CodeRabbit web-searched the dbt-bigquery source). dbt-bigquery uses `profile.project` as the execution_project; the SDK does not consult that env var. Rewrote the comment to document the actual behaviour + how the e2e test sidesteps it via a per-run profile rewrite in `tmp_path`. 4. README "Trying it out" cost wording conflated BigQuery scan cost (~$0.005) with Anthropic spend (~$0.13) and capped at <100 MB despite the test bumping `maximum_bytes_billed` to 1 GB for the materialised-sample CTAS (~200-500 MB). Separated the two cost lines. 5. README walkthrough as written would actually fail with the same billing-permission error we hit during the live run — the committed profile points at `bigquery-public-data`. Updated to show the working `cp` + profile-rewrite incantation (mirrors what the e2e test does internally). 6. CLAUDE.md `#10` bullet still mentioned "engineered literal / COALESCE'd columns" — Path A removed those (already fixed in prior commit; verify-and-resolve thread). 7. Plan fenced code block missing language hint (markdownlint MD040) — added `text`. 8. `test_austin_fixture_loads.py` comment said "nine columns" but the Path A SELECT exposes seven; updated. 9. `test_e2e_bigquery_smoke.py` docstring + assertion message still cited DEC-010 engineered columns; rewrote to reflect Path A's natural-NOT-NULL strategy. 10. `test_prompt_cache_stability.py` docstring referenced the pre-rotation hash; rewrote to point at the constant + log the rotation history without hardcoding a value. 11. `docs/cli-ops.md` cross-ref said "no setup beyond gcloud + Anthropic key" — explicitly names `GOOGLE_CLOUD_PROJECT` now and points at the new docs file. Already addressed (verify-and-resolve): 12. `LLMCacheTooSmallError` API removal flagged by Copilot — addressed in prior commit `bf5fe8b` (class + tests removed, docs + rules updated). Verify thread resolved. Validation: ruff + ruff format + pyright + pytest all green; 1463 passed, 13 deselected, 95.32% coverage. Live e2e re-run not required — changes are docs / comments / a defensive flag, not behaviour-changing for the live path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…roubleshooting (#33) Replaces the placeholder Quick start + "Trying it out" with a single five-step Quick start covering all of issue #11's AC: install, BigQuery + Anthropic auth, inline minimum signalforge.yml, first-run command, realistic expected-output table, and a five-row troubleshooting table. Updates the three doc/fixture references that anchor-linked to #trying-it-out so nothing breaks. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename PyPI distribution to `signalforge-dbt` (bare `signalforge` is held by an unrelated DSP package); import name and CLI command stay `signalforge`. - Bump `__version__` to `0.1.0rc1` for the test.pypi.org validation. A follow-up PR will flip to `0.1.0` once the test publish is verified. - Add `CHANGELOG.md` with the v0.1 line items. - Add `.github/workflows/publish.yml` mirroring clauditor's release flow: fires on `release: published`, prerelease -> test.pypi.org, full release -> pypi.org. Third-party actions SHA-pinned per `ci-supply-chain.md`. - Add `.github/PULL_REQUEST_TEMPLATE.md` for future contributions. - README / docs/cli-ops.md / CLAUDE.md updated to reflect the rename. Closes #12 (in two PRs; this one ships the plumbing + cuts rc1, follow-up flips to 0.1.0 and opens dev for v0.2 via 0.2.0.dev0). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughI can’t reliably reconstruct the required hidden review-stack artifact with all ~600 rangeIds exactly once within this response (the parser contract requires every provided rangeId to appear exactly once). Please ask me to regenerate the artifact and I will produce the full, valid hidden review-stack plus the visible walkthrough and change tables. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.claude/rules/ci-supply-chain.md (1)
1-68: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winUse issue-scoped rule file naming/splitting under
.claude/rules/.Please split or rename this to match the required
<issue>-<topic>.mdpattern (and keep issue-specific rules isolated per file).As per coding guidelines, “Distil each issue's architectural rules into a dedicated
.claude/rules/<issue>-<topic>.mdfile”.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/rules/ci-supply-chain.md around lines 1 - 68, The file .claude/rules/ci-supply-chain.md must be renamed/split to follow the required pattern and isolate the issue-specific rules; move its contents into one or more files named <issue>-<topic>.md (e.g., 1-ci-supply-chain.md or 1-ci-supply-chain-credentials.md) so each rule set maps to a single issue, update any internal references or indexes that list rule files to point to the new filenames, and remove or archive the original ci-supply-chain.md to avoid duplication.docs/manifest-loader-ops.md (1)
1-76: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winAdd explicit public-API and configuration sections to complete this ops doc.
This guide already covers operations and errors, but it should also explicitly document the public API functions/classes and a clear configuration/inputs section for the manifest loader surface.
As per coding guidelines, “Document all public-API functions, classes, and error hierarchies in dedicated ops documents … including operational references, error handling, and configuration sections”.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/manifest-loader-ops.md` around lines 1 - 76, The ops doc is missing explicit "Public API" and "Configuration/Inputs" sections; add a "Public API" section that lists and documents exported functions/classes (e.g., load(), MAX_MANIFEST_BYTES) and important types/methods (Manifest.get_model) and every public error class (ManifestNotFoundError, UnsupportedManifestVersionError, ModelNotFoundError, ModelDisabledError, ModelPathOutsideProjectError, ModelMissingSqlError) with short usage notes; add a "Configuration / Inputs" section that explains expected inputs to load() (manifest_path, any optional flags), the soft-size threshold behavior and how to override MAX_MANIFEST_BYTES for tests, memory sizing guidance (3× rule), and any environment or install requirements for schema versions (v9–v12 regeneration notes) so operators can run and configure the loader from the doc.
🟡 Minor comments (21)
plans/super/1-project-scaffolding.md-251-255 (1)
251-255:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd language identifiers to fenced code blocks.
Both fenced blocks are missing a language tag (MD040). Use
text/bashfor the callout snippet andtextfor the dependency graph block.Also applies to: 314-318
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plans/super/1-project-scaffolding.md` around lines 251 - 255, The two fenced code blocks around the "Status (v0.1, in progress)" callout and the dependency graph are missing language identifiers (MD040); update the first/callout fenced block to start with ```text or ```bash (use bash if you want shell highlighting for the pip command) and update the dependency-graph fenced block to start with ```text so both blocks include explicit language tags; locate the blocks by searching for the "Status (v0.1, in progress)" callout and the dependency graph block and add the appropriate language after the opening backticks.plans/super/22-temp-table-sample.md-263-263 (1)
263-263:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove extra spaces inside inline code spans on this line.
Line 263 triggers MD038 multiple times because several inline code spans include surrounding spaces. Tighten those code spans so markdownlint passes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plans/super/22-temp-table-sample.md` at line 263, Line 263 has inline code spans with extra spaces causing MD038; remove the spaces inside the backticks so the spans are tight (e.g. change ` materialise_sample ` to `materialise_sample`, ` run_id = ... ` to `run_id = ...`, `_sf_sample_<run_id>` etc.). Go through the text fragments referencing BigQueryAdapter, validate_identifier, CREATE TEMP TABLE, client.query(..., job_config=QueryJobConfig(...)), job.session_info.session_id, self._active_session_id, TableRef(...), run_test_sql, _active_session_id and __exit__ and ensure each inline code span has no surrounding spaces inside the backticks.plans/super/3-bigquery-adapter.md-119-140 (1)
119-140:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winEscape markdown metacharacters in table/code content to fix broken rendering and lint failures.
This range hits MD038/MD052/MD056 due to regex/type syntax and pipe characters inside table cells, plus MD040 for the dependency graph fence. Escape pipes (
\|), keep regex in safe inline-code form, and add a language tag (e.g.,text) to the fenced block.Also applies to: 676-691
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plans/super/3-bigquery-adapter.md` around lines 119 - 140, Escape markdown metacharacters inside the Markdown tables and fenced code blocks so the linter passes: backslash-escape pipes (`|` → `\|`) inside table cells, wrap regex/type snippets like r"^[A-Za-z_][A-Za-z0-9_]*$" and type expressions in inline code (e.g. `r"^[A-Za-z_][A-Za-z0-9_]*$"`, `int | float | str | None`) to avoid MD038/MD052/MD056, and add a language tag (e.g., ```text) to any fenced dependency/graph blocks to fix MD040; apply the same escaping and inline-code treatment to the other affected block that the comment references (the later table range). Ensure tokens like PartitionFilter, TableRef, ColumnStats, DbtProfileTarget, TestResult, and the SQL-snippet wrappers are preserved as code spans so pipes and punctuation don’t break table cells.docs/research/dbt-claude-technical-surface.md-494-494 (1)
494-494:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix markdown emphasis collision in cost math expression.
The
*operators are being parsed as emphasis markers (MD037). Replace with×(or escape\*) so markdownlint passes.Suggested patch
-A 200k-token project context cached at 1-hour TTL: write costs $3/M * 0.2M * 2 = $1.20 once. Subsequent reads in the same hour: $3/M * 0.2M * 0.1 = $0.06 per request. Run 50 PR reviews in an hour: $1.20 + 50 * $0.06 = $4.20 vs. uncached $3/M * 0.2M * 50 = $30.00. **86% savings.** +A 200k-token project context cached at 1-hour TTL: write costs $3/M × 0.2M × 2 = $1.20 once. Subsequent reads in the same hour: $3/M × 0.2M × 0.1 = $0.06 per request. Run 50 PR reviews in an hour: $1.20 + 50 × $0.06 = $4.20 vs. uncached $3/M × 0.2M × 50 = $30.00. **86% savings.**🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/research/dbt-claude-technical-surface.md` at line 494, The markdown math expression in the sentence beginning "A 200k-token project context cached at 1-hour TTL: write costs $3/M * 0.2M * 2 = $1.20 once..." uses `*` which is being interpreted as emphasis; update that expression (and the subsequent occurrences " $3/M * 0.2M * 0.1 " and " $3/M * 0.2M * 50 ") to use either the multiplication symbol `×` or escape the asterisks as `\*` so the inline math renders literally and passes markdownlint MD037..gitignore-36-39 (1)
36-39:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAllowlisted skill directories are not fully unignored recursively.
!.claude/skills/release-managerand!.claude/skills/review-agentskills-specunignore the directory entries, but nested files can still remain ignored. Add recursive unignore rules for contents.Suggested patch
.claude/skills/* !.claude/skills/release-manager !.claude/skills/review-agentskills-spec +!.claude/skills/release-manager/** +!.claude/skills/review-agentskills-spec/**🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.gitignore around lines 36 - 39, The current allowlist lines (!.claude/skills/release-manager and !.claude/skills/review-agentskills-spec) only unignore the directory entries but not their nested files; add recursive unignore rules by appending lines that unignore all contents under those directories (e.g. use patterns like !.claude/skills/release-manager/** and !.claude/skills/review-agentskills-spec/**) so files and subdirectories are not kept ignored while keeping the parent .claude/skills/* ignore rule..claude/rules/cli-layer.md-9-9 (1)
9-9:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix markdownlint violations in fenced/code-span formatting.
Line 9 uses an unlabeled fenced block (MD040), and Lines 45–46 include code-span spacing patterns flagged by MD038. Please normalize these to keep docs lint-clean.
Also applies to: 45-46
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/rules/cli-layer.md at line 9, The markdown has unlabeled fenced-code blocks and code-span spacing issues: add a language tag to the lone triple-backtick fenced block (replace ``` with ```markdown or ```text) to silence MD040, and normalize inline code spans flagged by MD038 by removing extraneous spaces inside backticks (change patterns like ` code ` to `code`) in the same document; ensure all fenced blocks use consistent language labels and all inline code uses no leading/trailing spaces within backticks.plans/super/2-manifest-loader.md-179-179 (1)
179-179:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd language identifiers to fenced code blocks.
The fenced blocks at Lines 179 and 482 are missing language tags (MD040). Please label them (for example,
text).Also applies to: 482-482
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plans/super/2-manifest-loader.md` at line 179, Two fenced code blocks use bare backticks (```) without language identifiers; update each ``` fence to include an explicit language tag (e.g., ```text or ```bash depending on the content) so they satisfy MD040. Locate the two occurrences of standalone ``` fences in the manifest loader markdown (the fenced blocks currently lacking a language tag) and add the appropriate language token to the opening fence for each block..claude/rules/warehouse-adapters.md-7-7 (1)
7-7:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winLabel the fenced code block language.
Line 7 opens a fenced block without a language identifier (MD040). Add a language tag (for example,
text).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/rules/warehouse-adapters.md at line 7, The Markdown fenced code block in .claude/rules/warehouse-adapters.md is missing a language identifier (MD040); update the opening triple-backtick to include a language tag (e.g., change ``` to ```text) so the code block is labeled properly.README.md-25-36 (1)
25-36:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd a language label to the fenced diagram block.
Line 25 triggers markdownlint MD040; please annotate the fence (e.g.,
text) to avoid lint noise.Suggested diff
-``` +```text ┌──────────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐ │ model.sql + │ -> │ LLM drafts │ -> │ Run tests │ -> │ Quality- │ │ manifest + │ │ candidate │ │ against the │ │ graded YAML │ │ project ctx │ │ artifacts │ │ warehouse │ │ + diff │ └──────────────┘ └─────────────┘ └──────────────┘ └──────────────┘ │ v Drop always-pass tests; drop tests that fail on known-clean data.</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@README.mdaround lines 25 - 36, Add a language label to the fenced diagram
block in README.md by changing the opening triple-backtick to include a language
(e.g., text) so the fence reads ```text; update the specific fenced block that
contains the ASCII diagram (the block starting with the box diagram and the
"Drop always-pass tests" lines) to silence markdownlint MD040.</details> </blockquote></details> <details> <summary>CLAUDE.md-69-75 (1)</summary><blockquote> `69-75`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Label the fenced pipeline block with a language.** Line 69 appears to violate markdownlint MD040; adding `text` keeps this file lint-safe. <details> <summary>Suggested diff</summary> ```diff -``` +```text model.sql + manifest + project ctx -> LLM drafts candidate artifacts -> run candidates against warehouse samples -> drop always-pass tests; drop tests that fail on known-clean data -> emit graded YAML + diff with per-artifact "why" ``` ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@CLAUDE.mdaround lines 69 - 75, The fenced code block in CLAUDE.md (the
block containing "model.sql + manifest + project ctx" and the subsequent
pipeline arrows) lacks a language label and triggers markdownlint MD040; fix it
by changing the opening fence fromtotext so the block is explicitly
labeled as plain text (i.e., update the opening triple-backticks of that
pipeline block to "```text").</details> </blockquote></details> <details> <summary>docs/cli-ops.md-201-209 (1)</summary><blockquote> `201-209`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Specify fenced-block languages for command transcripts.** Lines 201 and 214 open fenced blocks without a language (markdownlint MD040). <details> <summary>Suggested fix</summary> ```diff -``` +```text $ pwd /repo/dbt/my_project/models/marts @@ $ signalforge generate customers.sql --project-dir /repo/dbt/my_project ``` @@ -``` +```text $ signalforge generate models/marts/customers.sql --project-dir /tmp ERROR: --project-dir '/tmp' does not contain dbt_project.yml ↳ Remediation: Pass a path that points directly at a dbt project root (the directory containing dbt_project.yml). The flag is an absolute assertion; the CLI does not walk up from it. $ echo $? 1 ``` ``` </details> Also applies to: 214-222 <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/cli-ops.md` around lines 201 - 209, Update the fenced code blocks that contain command transcripts so they specify a language (use "text"); specifically, add "text" to the opening fences for the blocks that start with "$ pwd" / "$ signalforge generate customers.sql" and the block that starts with "$ signalforge generate models/marts/customers.sql" (the command-output examples shown in the diff) to satisfy markdownlint MD040. ``` </details> </blockquote></details> <details> <summary>docs/safety-ops.md-146-153 (1)</summary><blockquote> `146-153`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Add language tags to plain fenced blocks.** Lines 146 and 237 start fenced blocks without a language, which triggers markdownlint MD040. <details> <summary>Suggested fix</summary> ```diff -``` +```text *email email *phone phone *ssn ssn ``` @@ -``` +```text customer_ssn -> col_a3f29c61 ``` ``` </details> Also applies to: 237-239 <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@docs/safety-ops.mdaround lines 146 - 153, Add a language tag (e.g., "text")
to the plain fenced code blocks containing the PII example lines (the blocks
that start with "*email / email / *phone / phone / *ssn / ssn" and the later
block containing "customer_ssn -> col_a3f29c61") so they become fenced blocks
liketext ..., which resolves markdownlint MD040; update both occurrences
referenced in the diff.</details> </blockquote></details> <details> <summary>docs/research/dbt-tool-design-sketches.md-950-963 (1)</summary><blockquote> `950-963`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Specify a language for the fenced block.** Line 950 starts a fenced block without a language, which triggers markdownlint MD040. <details> <summary>Suggested fix</summary> ```diff -``` +```text models/ staging/insurance/stg_policies__active.sql staging/insurance/_stg_policies__schema.yml intermediate/int_active_policies_with_premiums.sql intermediate/int_premium_adjustments_calculated.sql marts/insurance/fct_premium_adjustments.sql tests/audit/ test_usp_calculate_monthly_premium_adjustments_row_counts.sql test_usp_calculate_monthly_premium_adjustments_value_diff.sql analyses/migration_notes/ usp_calculate_monthly_premium_adjustments.md # LLM-written migration commentary ``` ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/research/dbt-tool-design-sketches.md` around lines 950 - 963, The fenced code block beginning with "models/" (the markdown block at lines showing the project layout) lacks a language tag which triggers markdownlint MD040; fix it by adding an explicit language identifier (e.g., "text" or "bash") after the opening triple backticks so the block becomes a fenced code block with a language, ensuring the listing is properly linted and rendered. ``` </details> </blockquote></details> <details> <summary>docs/research/dbt-research-index.md-61-69 (1)</summary><blockquote> `61-69`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Fix the file-map root and add a fenced-block language.** Line 62 shows `docs/temp/`, which looks stale for this file set, and Line 61 opens a fence without a language (MD040). <details> <summary>Suggested fix</summary> ```diff -``` -docs/temp/ +```text +docs/research/ ├── dbt-research-index.md ← you are here ├── dbt-tooling-opportunity-report.md ← executive scan (start here) ├── dbt-pain-deep-dive.md ← user voice ├── dbt-ai-tools-deep-dive.md ← competitive landscape ├── dbt-tool-design-sketches.md ← three full designs + recommendation └── dbt-claude-technical-surface.md ← implementer reference ``` ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.In
@docs/research/dbt-research-index.mdaround lines 61 - 69, Update the fenced
code block in dbt-research-index.md by changing the root path string
"docs/temp/" to "docs/research/" and add a language specifier for the fenced
block (e.g., ```text) so the file-map is accurate and the MD040 lint rule is
satisfied; locate the fenced block that lists the tree (the block containing
"dbt-research-index.md", "dbt-tooling-opportunity-report.md", etc.), replace the
first line with "docs/research/" and add the language token after the opening
backticks.</details> </blockquote></details> <details> <summary>docs/cli-ops.md-255-257 (1)</summary><blockquote> `255-257`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Remove leading spaces inside inline code spans.** Line 256’s inline code includes leading spaces (`MD038`); keep indentation in prose, not inside backticks. <details> <summary>Suggested fix</summary> ```diff -- **Tier 1 and 3** — single line: `ERROR: <message>`, optionally - followed by ` ↳ Remediation: <text>` when the typed error +- **Tier 1 and 3** — single line: `ERROR: <message>`, optionally + followed by `↳ Remediation: <text>` when the typed error ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/cli-ops.md` around lines 255 - 257, Remove the leading spaces inside the inline code span for the remediation footer: replace the backticked snippet that currently reads "` ↳ Remediation: <text>`" with a no-leading-space version like "`↳ Remediation: <text>`" so the inline code spans (`ERROR: <message>` and the remediation footnote) do not contain leading spaces; update the prose accordingly. ``` </details> </blockquote></details> <details> <summary>.claude/rules/grade-layer.md-151-152 (1)</summary><blockquote> `151-152`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **`GradeBudgetExceededError` reservation note conflicts with current behavior docs.** This says “never raised in v0.1,” but the grade ops guide documents the “run did nothing” raise path. Please align the rule text to the implemented behavior. <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/rules/grade-layer.md around lines 151 - 152, Update the rule text to reflect the actual behavior: replace the claim "`GradeBudgetExceededError` — never raised in v0.1" with a statement that v0.1 can raise GradeBudgetExceededError on a hard "run did nothing" failure (matching the grade ops guide), and optionally note that v0.2 will continue to raise it for hard budget trips; also clarify that GradeThresholds is intended to become the canonical container and how it maps to existing GradeConfig fields and GradingReport.thresholds so readers can reconcile min_pass_rate/min_mean_score with the tuple shape. ``` </details> </blockquote></details> <details> <summary>.claude/rules/diff-renderer.md-181-182 (1)</summary><blockquote> `181-182`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **API signature in rule doc is missing `write_sidecar` kwarg.** The signature here conflicts with earlier default-on sidecar semantics documented in the same file. <details> <summary>🩹 Suggested edit</summary> ```diff -render_diff(model, candidate, prune_result, *, grading_report=None, existing_schema=None, config=None, output_path=None, sidecar_path=None, project_dir=None) -> DiffReport +render_diff(model, candidate, prune_result, *, grading_report=None, existing_schema=None, config=None, output_path=None, sidecar_path=None, write_sidecar=True, project_dir=None) -> DiffReport ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/rules/diff-renderer.md around lines 181 - 182, Update the documented API signature for render_diff to include the missing write_sidecar keyword parameter to match the precedent from grade_artifacts / prune_tests / draft_schema and the file's default-on sidecar semantics; modify the signature string `render_diff(model, candidate, prune_result, *, grading_report=None, existing_schema=None, config=None, output_path=None, sidecar_path=None, project_dir=None)` to include `write_sidecar=True` (or the appropriate default) and ensure any explanatory text mentions the default behavior. ``` </details> </blockquote></details> <details> <summary>docs/draft-ops.md-224-226 (1)</summary><blockquote> `224-226`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Stale statement: undersize cache blocks are no longer an error path.** This sentence contradicts the section above (undersize now drops cache marker and continues with a normal call). <details> <summary>🩹 Suggested edit</summary> ```diff -Both errors fire **before** any `messages.create` call, so an oversize -or undersize block never leaves a billable footprint. +The oversize error fires **before** any `messages.create` call, so an +oversize block leaves no billable footprint. Undersize blocks proceed +without cache markers. ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/draft-ops.md` around lines 224 - 226, The sentence stating "Both errors fire **before** any `messages.create` call, so an oversize or undersize block never leaves a billable footprint." is stale; update or remove it to reflect that undersize cache blocks are no longer an error path and instead drop the cache marker and continue with a normal `messages.create` call. Locate the sentence referencing `messages.create` and "undersize" in draft-ops.md and either reword it to explicitly state that oversize errors still fire before `messages.create` while undersize blocks now drop the cache marker and proceed, or delete the incorrect clause about undersize blocks. ``` </details> </blockquote></details> <details> <summary>docs/diff-ops.md-475-476 (1)</summary><blockquote> `475-476`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **CLI flag name appears inconsistent (`--render` vs `--format`).** Given the rest of this PR context, this likely should reference `--format` to match the actual CLI surface. <details> <summary>🩹 Suggested edit</summary> ```diff -Renderer selection (`--render=ansi|markdown|json`) overrides +Renderer selection (`--format=ansi|markdown|json`) overrides `config.render_kind`; ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/diff-ops.md` around lines 475 - 476, The docs mention a CLI flag `--render=ansi|markdown|json` which is inconsistent with the CLI surface in this PR; update the text to reference the actual flag `--format=ansi|markdown|json` so it matches `config.render_kind` and the `--no-color` override; specifically change the phrase "Renderer selection (`--render=ansi|markdown|json`) overrides `config.render_kind`; `--no-color` overrides" to use `--format` and verify surrounding mentions of "render" flags refer to the `--format` option and not a non-existent `--render`. ``` </details> </blockquote></details> <details> <summary>docs/prune-ops.md-16-19 (1)</summary><blockquote> `16-19`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Fix prune-layer differentiator wording (`grades` vs `prunes`).** This line describes the prune stage as “generates and grades,” which is stage-inaccurate and likely a copy/paste slip. <details> <summary>🩹 Suggested edit</summary> ```diff -This is the load-bearing differentiator (Architectural Commitment `#1` in -[`CLAUDE.md`](../CLAUDE.md)) — competitors generate; SignalForge -generates *and* grades. +This is the load-bearing differentiator (Architectural Commitment `#1` in +[`CLAUDE.md`](../CLAUDE.md)) — competitors generate; SignalForge +generates *and* prunes. ``` </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/prune-ops.md` around lines 16 - 19, The sentence "SignalForge generates *and* grades." in the prune-layer description is incorrect; update the wording so it accurately describes the prune stage (e.g., change "generates and grades" to "generates and prunes" or "generates, then prunes") to reflect that the stage prunes rather than grades. ``` </details> </blockquote></details> <details> <summary>docs/grade-ops.md-88-101 (1)</summary><blockquote> `88-101`: _⚠️ Potential issue_ | _🟡 Minor_ | _⚡ Quick win_ **Document `GradeBelowThresholdError` in the error hierarchy section.** This section omits a public error that is actively documented/used later in the file, so the hierarchy is incomplete. <details> <summary>🩹 Suggested doc patch</summary> ```diff - **`GradeOutputError`** — LLM-judge response failed parse or anchor-contract validation. Carries `violation_type: GradeOutputViolationType`. +- **`GradeBelowThresholdError`** — Raised when `fail_on_below_threshold=True` and the aggregate report fails threshold checks; carries `pass_rate`, `mean_score`, `min_pass_rate`, `min_mean_score`, and `aggregate_complete`. - **`GradeAuditWriteError`** — Fail-closed audit-write failure (`OSError` / `PermissionError` / encoding / `fsync` / symlink containment). Aborts the run; original cause exposed via `.cause` and `__cause__`. ``` </details> As per coding guidelines: “`docs/**/*.md`: Document all public-API functions, classes, and error hierarchies … including operational references, error handling, and configuration sections”. <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/grade-ops.md` around lines 88 - 101, Add a new bullet for GradeBelowThresholdError to the error hierarchy list (consistent with the existing entries like GradeError, GradeOutputError, GradeBudgetExceededError) describing that GradeBelowThresholdError is a public error raised when a run’s aggregate score falls below the configured passing threshold; mention any relevant attributes/semantics used later in the docs such as that it relates to aggregate/pass thresholds on GradingReport and that it should surface a default_remediation via __str__ like other Grade* errors. Ensure the wording and formatting match the surrounding bullets (name in bold/code style, short one-line description, note about preserved cause/default_remediation if applicable) so the hierarchy is complete and consistent with later references to GradeBelowThresholdError. ``` </details> </blockquote></details> </blockquote></details> <details> <summary>🧹 Nitpick comments (3)</summary><blockquote> <details> <summary>.github/workflows/publish.yml (2)</summary><blockquote> `27-27`: _💤 Low value_ **Verify prerelease conditional syntax consistency.** Line 27 uses `if: github.event.release.prerelease` (simple expression) while line 52 uses `if: ${{ !github.event.release.prerelease }}` (with `${{ }}` wrapper). While both are valid GitHub Actions syntax, the wrapper is redundant in `if:` contexts. For consistency and following GitHub's current recommendations, consider using the simpler syntax without the wrapper. <details> <summary>♻️ Proposed consistency fix</summary> ```diff publish-pypi: name: Publish to PyPI - if: ${{ !github.event.release.prerelease }} + if: '!github.event.release.prerelease' runs-on: ubuntu-latest ``` Note: The quotes around the expression are needed when starting with `!` to prevent YAML from treating it as a tag. </details> <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/publish.yml at line 27, Update the prerelease conditional expressions for consistency: change the wrapped expression used in the non-prerelease branch (if: ${{ !github.event.release.prerelease }}) to the simpler unwrapped form (if: !github.event.release.prerelease) and add quotes around it (if: "!github.event.release.prerelease") to avoid YAML parsing issues; ensure the prerelease branch remains as if: github.event.release.prerelease so both branches use the same unwrapped syntax style. ``` </details> --- `42-42`: _💤 Low value_ **Consider more specific build version pinning.** Line 42 (and 67) pins `build==1.2.*`, which will automatically pull patch updates. While this provides some stability, the PR objectives mention this is a release workflow where reproducibility is critical. Consider whether full pinning (e.g., `build==1.2.0`) would be more appropriate for release builds to ensure identical artifacts across rebuilds. This is a trade-off between receiving patch fixes automatically vs. maximum reproducibility. The current approach is reasonable, but document the choice if you prefer maximum reproducibility. <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/publish.yml at line 42, The workflow currently installs build with a wildcard patch pin (`pip install build==1.2.*`) which allows patch updates and reduces reproducibility; change the install steps that use `build==1.2.*` (both occurrences) to a fully pinned version like `build==1.2.0` for exact reproducibility for release builds, or alternatively add a short comment in the workflow explaining the deliberate choice to allow patch updates if you prefer automatic fixes instead of strict pinning. ``` </details> </blockquote></details> <details> <summary>docs/warehouse-adapter-ops.md (1)</summary><blockquote> `1-540`: _💤 Low value_ **Excellent comprehensive operations guide!** This documentation thoroughly meets all coding guidelines requirements for `docs/**/*.md`. The guide successfully documents: - ✅ Public-API functions and classes (WarehouseAdapter, TableRef, PartitionFilter, ColumnStats) - ✅ Complete error hierarchy with remediation guidance (19 error classes in the reference table) - ✅ Operational references (quick start, integration tests, debugging, manual recovery) - ✅ Error handling patterns and fail-loud thresholds - ✅ Configuration details (dbt profile resolution, cost defaults, auth methods) The structure is logical and navigable, with clear section headers and appropriate cross-references to related docs. Code examples cover the full spectrum from quick start to advanced debugging. The mixing of v0.1 and v0.2 features is clearly labeled throughout, which helps users understand the roadmap while keeping all adapter documentation centralized. --- **Optional style refinements** (from static analysis): Three minor wordiness/style suggestions that could marginally improve clarity, but are entirely optional: - Line 175: "in conjunction with" → "with" (shorter but current phrasing is fine for technical docs) - Line 453: "by accident" → "accidentally" (shorter but current is clear) - Line 530: "style IDs" → "style identifiers" (slightly clearer context) <details> <summary>🤖 Prompt for AI Agents</summary> ``` Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/warehouse-adapter-ops.md` around lines 1 - 540, Replace three optional wordy phrases in the document: change the phrase "in conjunction with" to "with" where it appears in the TABLESAMPLE discussion (search for the exact phrase "in conjunction with a partition filter"); change "by accident" to "accidentally" in the Integration tests section (search for "by accident"); and change "style IDs" to "style identifiers" in the v0.2 follow-ups section (search for "style IDs"); keep all other text and formatting unchanged. ``` </details> </blockquote></details> </blockquote></details> <details> <summary>🤖 Prompt for all review comments with AI agents</summary>Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.Inline comments:
In @.claude/skills/release-manager/SKILL.md:
- Around line 3-4: The release skill is still referencing "clauditor" and
"clauditor-eval" identifiers; update all occurrences of those package/repo names
to the correct repository/package identifiers for this project (search for the
literal strings "clauditor" and "clauditor-eval" and replace them), and ensure
the description and compatibility fields (the YAML keys description and
compatibility) and any release target mentions (TestPyPI vs PyPI) reflect the
correct target repo and package names; apply the same replacements in the other
referenced blocks (around the earlier listed occurrences) and verify any
CLI/gh/git invocation examples or paths in the SKILL.md content are updated to
use the correct repo/package names.In @.github/workflows/ci.yml:
- Around line 4-7: The CI workflow only triggers pull_request events for the dev
branch, so PRs targeting main skip pre-merge checks; update the workflow's
pull_request triggers by adding main to the branches list (i.e., modify the
pull_request: branches entry so it includes both dev and main) so pull requests
to main run the CI just like dev; ensure the push triggers remain unchanged.
Outside diff comments:
In @.claude/rules/ci-supply-chain.md:
- Around line 1-68: The file .claude/rules/ci-supply-chain.md must be
renamed/split to follow the required pattern and isolate the issue-specific
rules; move its contents into one or more files named -.md (e.g.,
1-ci-supply-chain.md or 1-ci-supply-chain-credentials.md) so each rule set maps
to a single issue, update any internal references or indexes that list rule
files to point to the new filenames, and remove or archive the original
ci-supply-chain.md to avoid duplication.In
@docs/manifest-loader-ops.md:
- Around line 1-76: The ops doc is missing explicit "Public API" and
"Configuration/Inputs" sections; add a "Public API" section that lists and
documents exported functions/classes (e.g., load(), MAX_MANIFEST_BYTES) and
important types/methods (Manifest.get_model) and every public error class
(ManifestNotFoundError, UnsupportedManifestVersionError, ModelNotFoundError,
ModelDisabledError, ModelPathOutsideProjectError, ModelMissingSqlError) with
short usage notes; add a "Configuration / Inputs" section that explains expected
inputs to load() (manifest_path, any optional flags), the soft-size threshold
behavior and how to override MAX_MANIFEST_BYTES for tests, memory sizing
guidance (3× rule), and any environment or install requirements for schema
versions (v9–v12 regeneration notes) so operators can run and configure the
loader from the doc.
Minor comments:
In @.claude/rules/cli-layer.md:
- Line 9: The markdown has unlabeled fenced-code blocks and code-span spacing
issues: add a language tag to the lone triple-backtick fenced block (replacewithmarkdown or ```text) to silence MD040, and normalize inline code spans
flagged by MD038 by removing extraneous spaces inside backticks (change patterns
likecodeto `code`) in the same document; ensure all fenced blocks use
consistent language labels and all inline code uses no leading/trailing spaces
within backticks.In @.claude/rules/diff-renderer.md:
- Around line 181-182: Update the documented API signature for render_diff to
include the missing write_sidecar keyword parameter to match the precedent from
grade_artifacts / prune_tests / draft_schema and the file's default-on sidecar
semantics; modify the signature stringrender_diff(model, candidate, prune_result, *, grading_report=None, existing_schema=None, config=None, output_path=None, sidecar_path=None, project_dir=None)to include
write_sidecar=True(or the appropriate default) and ensure any explanatory
text mentions the default behavior.In @.claude/rules/grade-layer.md:
- Around line 151-152: Update the rule text to reflect the actual behavior:
replace the claim "GradeBudgetExceededError— never raised in v0.1" with a
statement that v0.1 can raise GradeBudgetExceededError on a hard "run did
nothing" failure (matching the grade ops guide), and optionally note that v0.2
will continue to raise it for hard budget trips; also clarify that
GradeThresholds is intended to become the canonical container and how it maps to
existing GradeConfig fields and GradingReport.thresholds so readers can
reconcile min_pass_rate/min_mean_score with the tuple shape.In @.claude/rules/warehouse-adapters.md:
- Line 7: The Markdown fenced code block in .claude/rules/warehouse-adapters.md
is missing a language identifier (MD040); update the opening triple-backtick to
include a language tag (e.g., changetotext) so the code block is
labeled properly.In @.gitignore:
- Around line 36-39: The current allowlist lines
(!.claude/skills/release-manager and !.claude/skills/review-agentskills-spec)
only unignore the directory entries but not their nested files; add recursive
unignore rules by appending lines that unignore all contents under those
directories (e.g. use patterns like !.claude/skills/release-manager/** and
!.claude/skills/review-agentskills-spec/**) so files and subdirectories are not
kept ignored while keeping the parent .claude/skills/* ignore rule.In
@CLAUDE.md:
- Around line 69-75: The fenced code block in CLAUDE.md (the block containing
"model.sql + manifest + project ctx" and the subsequent pipeline arrows) lacks a
language label and triggers markdownlint MD040; fix it by changing the opening
fence fromtotext so the block is explicitly labeled as plain text
(i.e., update the opening triple-backticks of that pipeline block to "```text").In
@docs/cli-ops.md:
- Around line 201-209: Update the fenced code blocks that contain command
transcripts so they specify a language (use "text"); specifically, add "text" to
the opening fences for the blocks that start with "$ pwd" / "$ signalforge
generate customers.sql" and the block that starts with "$ signalforge generate
models/marts/customers.sql" (the command-output examples shown in the diff) to
satisfy markdownlint MD040.- Around line 255-257: Remove the leading spaces inside the inline code span for
the remediation footer: replace the backticked snippet that currently reads "↳ Remediation: <text>" with a no-leading-space version like "↳ Remediation: <text>" so the inline code spans (ERROR: <message>and the remediation
footnote) do not contain leading spaces; update the prose accordingly.In
@docs/diff-ops.md:
- Around line 475-476: The docs mention a CLI flag
--render=ansi|markdown|json
which is inconsistent with the CLI surface in this PR; update the text to
reference the actual flag--format=ansi|markdown|jsonso it matches
config.render_kindand the--no-coloroverride; specifically change the
phrase "Renderer selection (--render=ansi|markdown|json) overrides
config.render_kind;--no-coloroverrides" to use--formatand verify
surrounding mentions of "render" flags refer to the--formatoption and not a
non-existent--render.In
@docs/draft-ops.md:
- Around line 224-226: The sentence stating "Both errors fire before any
messages.createcall, so an oversize or undersize block never leaves a
billable footprint." is stale; update or remove it to reflect that undersize
cache blocks are no longer an error path and instead drop the cache marker and
continue with a normalmessages.createcall. Locate the sentence referencing
messages.createand "undersize" in draft-ops.md and either reword it to
explicitly state that oversize errors still fire beforemessages.createwhile
undersize blocks now drop the cache marker and proceed, or delete the incorrect
clause about undersize blocks.In
@docs/grade-ops.md:
- Around line 88-101: Add a new bullet for GradeBelowThresholdError to the error
hierarchy list (consistent with the existing entries like GradeError,
GradeOutputError, GradeBudgetExceededError) describing that
GradeBelowThresholdError is a public error raised when a run’s aggregate score
falls below the configured passing threshold; mention any relevant
attributes/semantics used later in the docs such as that it relates to
aggregate/pass thresholds on GradingReport and that it should surface a
default_remediation via str like other Grade* errors. Ensure the wording and
formatting match the surrounding bullets (name in bold/code style, short
one-line description, note about preserved cause/default_remediation if
applicable) so the hierarchy is complete and consistent with later references to
GradeBelowThresholdError.In
@docs/prune-ops.md:
- Around line 16-19: The sentence "SignalForge generates and grades." in the
prune-layer description is incorrect; update the wording so it accurately
describes the prune stage (e.g., change "generates and grades" to "generates and
prunes" or "generates, then prunes") to reflect that the stage prunes rather
than grades.In
@docs/research/dbt-claude-technical-surface.md:
- Line 494: The markdown math expression in the sentence beginning "A 200k-token
project context cached at 1-hour TTL: write costs $3/M * 0.2M * 2 = $1.20
once..." uses*which is being interpreted as emphasis; update that expression
(and the subsequent occurrences " $3/M * 0.2M * 0.1 " and " $3/M * 0.2M * 50 ")
to use either the multiplication symbol×or escape the asterisks as\*so
the inline math renders literally and passes markdownlint MD037.In
@docs/research/dbt-research-index.md:
- Around line 61-69: Update the fenced code block in dbt-research-index.md by
changing the root path string "docs/temp/" to "docs/research/" and add a
language specifier for the fenced block (e.g., ```text) so the file-map is
accurate and the MD040 lint rule is satisfied; locate the fenced block that
lists the tree (the block containing "dbt-research-index.md",
"dbt-tooling-opportunity-report.md", etc.), replace the first line with
"docs/research/" and add the language token after the opening backticks.In
@docs/research/dbt-tool-design-sketches.md:
- Around line 950-963: The fenced code block beginning with "models/" (the
markdown block at lines showing the project layout) lacks a language tag which
triggers markdownlint MD040; fix it by adding an explicit language identifier
(e.g., "text" or "bash") after the opening triple backticks so the block becomes
a fenced code block with a language, ensuring the listing is properly linted and
rendered.In
@docs/safety-ops.md:
- Around line 146-153: Add a language tag (e.g., "text") to the plain fenced
code blocks containing the PII example lines (the blocks that start with "*email
/ email / *phone / phone / *ssn / ssn" and the later block containing
"customer_ssn -> col_a3f29c61") so they become fenced blocks like ```text ...the diff. In `@plans/super/1-project-scaffolding.md`: - Around line 251-255: The two fenced code blocks around the "Status (v0.1, in progress)" callout and the dependency graph are missing language identifiers (MD040); update the first/callout fenced block to start with ```text or ```bash (use bash if you want shell highlighting for the pip command) and update the dependency-graph fenced block to start with ```text so both blocks include explicit language tags; locate the blocks by searching for the "Status (v0.1, in progress)" callout and the dependency graph block and add the appropriate language after the opening backticks. In `@plans/super/2-manifest-loader.md`: - Line 179: Two fenced code blocks use bare backticks (```) without language identifiers; update each ``` fence to include an explicit language tag (e.g., ```text or ```bash depending on the content) so they satisfy MD040. Locate the two occurrences of standalone ``` fences in the manifest loader markdown (the fenced blocks currently lacking a language tag) and add the appropriate language token to the opening fence for each block. In `@plans/super/22-temp-table-sample.md`: - Line 263: Line 263 has inline code spans with extra spaces causing MD038; remove the spaces inside the backticks so the spans are tight (e.g. change ` materialise_sample ` to `materialise_sample`, ` run_id = ... ` to `run_id = ...`, `_sf_sample_<run_id>` etc.). Go through the text fragments referencing BigQueryAdapter, validate_identifier, CREATE TEMP TABLE, client.query(..., job_config=QueryJobConfig(...)), job.session_info.session_id, self._active_session_id, TableRef(...), run_test_sql, _active_session_id and __exit__ and ensure each inline code span has no surrounding spaces inside the backticks. In `@plans/super/3-bigquery-adapter.md`: - Around line 119-140: Escape markdown metacharacters inside the Markdown tables and fenced code blocks so the linter passes: backslash-escape pipes (`|` → `\|`) inside table cells, wrap regex/type snippets like r"^[A-Za-z_][A-Za-z0-9_]*$" and type expressions in inline code (e.g. `r"^[A-Za-z_][A-Za-z0-9_]*$"`, `int | float | str | None`) to avoid MD038/MD052/MD056, and add a language tag (e.g., ```text) to any fenced dependency/graph blocks to fix MD040; apply the same escaping and inline-code treatment to the other affected block that the comment references (the later table range). Ensure tokens like PartitionFilter, TableRef, ColumnStats, DbtProfileTarget, TestResult, and the SQL-snippet wrappers are preserved as code spans so pipes and punctuation don’t break table cells. In `@README.md`: - Around line 25-36: Add a language label to the fenced diagram block in README.md by changing the opening triple-backtick to include a language (e.g., text) so the fence reads ```text; update the specific fenced block that contains the ASCII diagram (the block starting with the box diagram and the "Drop always-pass tests" lines) to silence markdownlint MD040. --- Nitpick comments: In @.github/workflows/publish.yml: - Line 27: Update the prerelease conditional expressions for consistency: change the wrapped expression used in the non-prerelease branch (if: ${{ !github.event.release.prerelease }}) to the simpler unwrapped form (if: !github.event.release.prerelease) and add quotes around it (if: "!github.event.release.prerelease") to avoid YAML parsing issues; ensure the prerelease branch remains as if: github.event.release.prerelease so both branches use the same unwrapped syntax style. - Line 42: The workflow currently installs build with a wildcard patch pin (`pip install build==1.2.*`) which allows patch updates and reduces reproducibility; change the install steps that use `build==1.2.*` (both occurrences) to a fully pinned version like `build==1.2.0` for exact reproducibility for release builds, or alternatively add a short comment in the workflow explaining the deliberate choice to allow patch updates if you prefer automatic fixes instead of strict pinning. In `@docs/warehouse-adapter-ops.md`: - Around line 1-540: Replace three optional wordy phrases in the document: change the phrase "in conjunction with" to "with" where it appears in the TABLESAMPLE discussion (search for the exact phrase "in conjunction with a partition filter"); change "by accident" to "accidentally" in the Integration tests section (search for "by accident"); and change "style IDs" to "style identifiers" in the v0.2 follow-ups section (search for "style IDs"); keep all other text and formatting unchanged.🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
- ci.yml: add `main` to the `pull_request` branches so release PRs into main get pre-merge CI (PR #34 itself bypassed CI for this reason). - release-manager skill: rewrite for SignalForge's actual release flow. Was forked from clauditor and never adapted — pure find/replace would have left it broken (assumed uv/uv.lock, branch-based publish routing, pyproject `version` field instead of __init__.py __version__, .devN suffix convention instead of rcN). Now matches: * pip + `python -m build`, `uvx twine check` for verification * version source = src/signalforge/__init__.py __version__ * publish.yml routing = GitHub Release prerelease flag (not branch) * package name `signalforge-dbt` throughout; repo `wjduenow/SignalForge` * rcN pre-release convention * canonical validation = ruff + pyright + pytest - CHANGELOG.md: add `[Unreleased]` placeholder section so the skill's Step 1b promotion (`[Unreleased]` -> `[X.Y.Z]`) has a real target on the next release. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
wjduenow
added a commit
that referenced
this pull request
May 11, 2026
* Initial commit * 12: v0.1.0 release plumbing (rc1 for test.pypi.org dry run) - Rename PyPI distribution to `signalforge-dbt` (bare `signalforge` is held by an unrelated DSP package); import name and CLI command stay `signalforge`. - Bump `__version__` to `0.1.0rc1` for the test.pypi.org validation. A follow-up PR will flip to `0.1.0` once the test publish is verified. - Add `CHANGELOG.md` with the v0.1 line items. - Add `.github/workflows/publish.yml` mirroring clauditor's release flow: fires on `release: published`, prerelease -> test.pypi.org, full release -> pypi.org. Third-party actions SHA-pinned per `ci-supply-chain.md`. - Add `.github/PULL_REQUEST_TEMPLATE.md` for future contributions. - README / docs/cli-ops.md / CLAUDE.md updated to reflect the rename. Closes #12 (in two PRs; this one ships the plumbing + cuts rc1, follow-up flips to 0.1.0 and opens dev for v0.2 via 0.2.0.dev0). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 12: address CodeRabbit review on PR #34 - ci.yml: add `main` to the `pull_request` branches so release PRs into main get pre-merge CI (PR #34 itself bypassed CI for this reason). - release-manager skill: rewrite for SignalForge's actual release flow. Was forked from clauditor and never adapted — pure find/replace would have left it broken (assumed uv/uv.lock, branch-based publish routing, pyproject `version` field instead of __init__.py __version__, .devN suffix convention instead of rcN). Now matches: * pip + `python -m build`, `uvx twine check` for verification * version source = src/signalforge/__init__.py __version__ * publish.yml routing = GitHub Release prerelease flag (not branch) * package name `signalforge-dbt` throughout; repo `wjduenow/SignalForge` * rcN pre-release convention * canonical validation = ruff + pyright + pytest - CHANGELOG.md: add `[Unreleased]` placeholder section so the skill's Step 1b promotion (`[Unreleased]` -> `[X.Y.Z]`) has a real target on the next release. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 36: super plan for `signalforge generate --estimate` Adds the Phase 4 plan for a pre-flight cost preview flag built on the already-wired `count_tokens` SDK seam plus a new BigQuery `dry_run` adapter method. 15 DECs cover price-table shape, partial- failure degrade (mirrors prune DEC-009), full-prelude ordering, and the 5-surface parity gate. 8 stories: pricing module, warehouse ABC extension, estimate engine + renderer, CLI wiring, docs, QG, patterns. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 36: mark plan as published in PR #63 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 36: devolve plan to beads (epic bd_1-scaffolding-bmz) 8 stories created with 16 dependency edges wired. US-001 ready; US-008 (Patterns & Memory) waits on US-007 (Quality Gate). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.1: US-001 add signalforge.llm.pricing module Public price table + ModelPricing dataclass + lookup() + EstimateUnknownModelError (tier 2). Three v0.1 SKUs: claude-sonnet-4-6, claude-opus-4-7, claude-haiku-4-5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.2: US-002 add WarehouseAdapter.estimate_query_bytes + BigQuery dryRun New ABC method (default raises EstimateNotSupportedError); BigQueryAdapter override uses QueryJobConfig(dry_run=True), reads total_bytes_processed. Mirrors materialise_sample precedent (#22) for graceful non-BQ-adapter degrade. EstimateNotSupportedError mapped at tier 3. FakeBigQueryClient gains expect_dry_run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.3: US-003 add signalforge.cli._estimate engine Pure-function estimate(...) returning frozen EstimateReport. Issues exactly one count_tokens per (draft + per-criterion rep) and one dry_run query. NEVER calls messages.create. Warehouse failures captured in warehouse_unavailable_reason via degrade-rather-than-fail (mirrors prune DEC-009); all other errors propagate. Single end-of-run INFO log via lazy-format JSON. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.4: US-004 add estimate text renderer + snapshot fixtures Pure-function render(report) returning the locked DEC-007 three-section shape. Snapshot fixtures pin happy + partial-failure paths byte-for-byte. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.5: US-005 wire --estimate flag in signalforge generate Adds --estimate to the existing --write/--dry-run mutex group; the cmd_generate handler short-circuits after the full prelude to _estimate.estimate(...) + _estimate.render(...) on stdout, returning 0. AC-4 pinned: zero messages.create calls, >=1 count_tokens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.6: US-006 docs + 5-surface parity for --estimate Adds --estimate to docs/cli-ops.md § Flag reference (surface 3 of the 5-surface parity rule). Extends CLAUDE.md § Public API surface with the pricing module + warehouse estimate_query_bytes additions. README Quick Start gets a one-line pointer. Refinement log notes the parity gate is now reconciled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.7: Quality gate — fix bugs from 4 code-review passes Pass 1 (correctness): - BLOCKER B-1: thread --mode/--scope/--sample-strategy CLI overrides into the --estimate short-circuit (DEC-009). Without these, an operator saying --scope full would have got a config-file scope estimate. - I-3: drop the mutable _PRICES_MUTABLE binding after wrapping in MappingProxyType; orphan the dict so it can't be mutated via the proxy. Pass 2 (security): - I-1: route the manifest-supplied column name through validate_identifier before quoting in the representative SQL builder; defense-in-depth against manifest-injection. - I-2: add custom CriterionEstimate.__repr__ to omit the criterion_text_truncated field (mirrors prune-engine.md DEC-022). Pass 3 (5-surface parity drift): - DRIFT-1: correct the DEC-008 mutex-tier wording (argparse exits via SystemExit(2) directly; no CliInputError raised). Pinned in the plan and the architecture-review table. - DRIFT-3: fix stale test path references in DEC-007 and docs/cli-ops.md (test_estimate_output_snapshot.py -> test_estimate_render.py). - DRIFT-4: enumerate "safety + draft + prune + grade + diff configs" verbatim in the cmd_generate docstring (was "every stage config"). Pass 4 (cross-stage propagation): - B-3: add caplog-based test pinning the DEC-005 stderr WARNING shape. Without it, a refactor that silently drops the warning leaves operators with no out-of-band signal that the run was degraded. Validation: ruff/format/pyright clean. 1516 tests passing (one new), 95.17% coverage, subprocess smoke green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bd_1-scaffolding-bmz.8: US-008 patterns & memory rules updates Extends .claude/rules/warehouse-adapters.md with the new estimate_query_bytes ABC method (issue #36, v0.2) and elevates it as the second non-BQ-adapter-graceful-degrade method under the ABC (alongside materialise_sample). Documents the fake-parity queue- isolation rule. Extends .claude/rules/cli-layer.md with a new section "Estimate-style commands degrade on supplementary sub-stage failures (DEC-005 of #36)" — generalises the QG pass-4 lesson: pin BOTH the report-field AND the WARNING via tests. See-Also pointers to prune-engine.md DEC-009 and warehouse-adapters.md cleanup-boundary fail-soft. Plan phase advanced to "complete". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * 36: address CodeRabbit review on PR #63 Clarify EstimateUnknownModelError inheritance-vs-mapping precedence in the plan (CodeRabbit comment on line 160): the MRO walk in map_exception_to_exit_code finds the concrete-class entry (tier 2) before the parent LLMError tier-3 default would apply, so the tier-2 mapping is deterministic. Other 8 CodeRabbit comments documented as false positives in the summary comment on the PR (out-of-scope files from the origin/dev merge, pre-existing markdownlint nits, already-fixed in QG pass 3). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires up the v0.1.0 release: PyPI publish workflow (clauditor-style), CHANGELOG, PR template, distribution rename to `signalforge-dbt`, and a `0.1.0rc1` cut for a test.pypi.org dry run before the real publish.
Mirrors clauditor's flow: GitHub Release `published` event triggers `publish.yml`; the prerelease checkbox routes to test.pypi.org vs prod pypi.org via separate trusted-publisher environments.
Distribution name is `signalforge-dbt` because the bare `signalforge` on PyPI is held by an unrelated DSP package. Import name and CLI command remain `signalforge` — no source-code surface changes.
This PR also includes a one-time merge commit aligning `main` (previously orphan, Initial commit only) with `dev`'s v0.1 history so the release PR can open.
Closes #12 once the follow-up (rc1 → 0.1.0 + dev → 0.2.0.dev0) lands.
Changes
Release sequence after this PR merges
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit