2: dbt manifest loader (plan) - #15
Merged
Merged
Conversation
- plans/super/2-manifest-loader.md: full /super-plan output (Phases 1-4) with 17 decisions and 9 stories (US-001..US-009 incl. Quality Gate and Patterns & Memory). - docs/research/: pinned snapshots of the seven dbt-prefixed research files from clauditor's gitignored docs/temp/, so contributors outside the maintainer's machine can read the references the plan cites (DEC-006). No code yet -- this is the planning PR. Implementation lands per the beads task graph after approval. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Epic bd_1-scaffolding-28p + 9 tasks created with 10 dependency edges matching the story graph. bd ready returns only .1 (US-001). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pytest markers - pydantic>=2.5,<3 as the first runtime dep (DEC-001). - dbt-core>=1.8,<2 in [dev] for fixture regeneration (DEC-009). - pytest markers unit/integration/error declared (DEC-015). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- src/signalforge/manifest/{__init__.py, errors.py}: seven-class hierarchy
rooted at ManifestError, with remediation: str rendered in __str__
(DEC-013, DEC-014).
- tests/manifest/test_errors.py: ≥6 fail-capable unit tests covering
hierarchy, default remediation, override, and base-class catching.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…dium, 5 error paths) - tests/fixtures/dbt_project_small/: 4-5 model dbt project + manifest_v9/10/11/12.json (DEC-012). - tests/fixtures/dbt_project_medium/: ~50-model synthesised project + manifest_v12.json. - tests/fixtures/error_paths/: malformed/missing_version_url/unsupported_v99/disabled_only/empty_raw_code. - tests/fixtures/regenerate.sh + README.md (DEC-009 multi-version regen recipe). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…schemas, medium, 5 error paths)
- src/signalforge/manifest/models.py: Manifest, Model, Column, Ref, Config, DependsOn — frozen + extra=ignore + populate_by_name. Validators on unique_id (must start "model.") and raw_code (strip-to-None) per DEC-016. columns_list property on Model. Nested Config per DEC-011. - tests/manifest/test_models.py: TDD tests parametrised across the four committed v9/v10/v11/v12 small manifests + extra="forbid" drift detector. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- src/signalforge/manifest/loader.py: load(), _detect_version, _canonicalise_path with .resolve()+is_relative_to() symlink hardening (DEC-007), MAX_MANIFEST_BYTES soft warning (DEC-008), manifest_path override (DEC-010), get_model/iter_models/schema_version free functions backed by lazily-built unique_id and file-path indexes. - src/signalforge/manifest/models.py: thin method wrappers on Manifest delegating to loader free functions (deferred import, no cycle). - tests/manifest/test_loader.py: >=16 fail-capable tests including the 7 Phase 2 regression tests, symlink hardening, 200MB warning, all error paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- src/signalforge/manifest/__init__.py: re-exports load, Manifest, Model, and all error classes; declares __all__ (DEC-017). - tests/manifest/test_public_api.py: programmatic importability check, Pydantic isinstance checks, error-hierarchy preservation, and explicit guards against accidental promotion of internals. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…uide, CONTRIBUTING) - docs/research/README.md: snapshot framing for the seven vendored dbt research files (DEC-006). - docs/manifest-loader-ops.md: memory profile table, soft 200MB warning posture (DEC-008), multi-version regen cross-link (DEC-009/DEC-012), supported-version table, error-class quick reference. - CONTRIBUTING.md: Test markers + Regenerating fixtures subsections (DEC-015 / DEC-009). - README.md: v0.1 status callout updated to reflect library-first delivery. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sses 1-4 Pass 2 (adversarial) surfaced two real bugs in the path-hardening logic; both are now fixed with regression tests that fail before the fix. 1. Default target/manifest.json bypassed _canonicalise_path. A symlink at target/manifest.json -> /etc/passwd would have been silently followed. Fix: route the default path through _canonicalise_path the same way the manifest_path override is handled. (DEC-007 hardening, applies uniformly.) 2. Path.resolve() raises RuntimeError on symlink cycles regardless of strict=. _canonicalise_path leaked that bare RuntimeError instead of the typed ModelPathOutsideProjectError our hierarchy promises. Fix: wrap both resolve() calls in try/except RuntimeError and re-raise with the same remediation surface. Three new fail-capable regression tests: - test_default_manifest_path_symlink_escape_is_rejected - test_symlink_loop_in_default_path_is_rejected - test_symlink_loop_in_explicit_manifest_path_is_rejected Pass 3 (test correctness) flagged one marker mis-mark: test_oversize_manifest_emits_warning was @pytest.mark.unit but reads a real fixture file; corrected to @pytest.mark.integration. Validation: ruff/format/pyright all green; pytest 67 passed (64 → 67, +3 regression tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- .claude/rules/manifest-readers.md (new): Pydantic v2 frozen+extra=ignore for external-format readers; the three symlink-resolution traps caught by issue #2's pass-2 review (Path.relative_to vs resolve, RuntimeError on cycles, default paths must be canonicalised too); ManifestError + remediation: str pattern; no logging/metrics in stage-0 modules. - .claude/rules/testing-signal.md: appended sections on fixture regeneration via ephemeral uvx and the extra="forbid" drift detector pattern (DEC-005, DEC-009, DEC-012, DEC-017). - CLAUDE.md: marked issue #2 shipped; added "Public API surface (v0.1)" section listing signalforge.manifest as the first stable surface; cross-link to docs/manifest-loader-ops.md. Auto-memory entry added (outside the repo): the editable-install race that bit US-002's worker-in-worktree pattern — orchestrators must reinstall from the merged feature worktree before validating. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Implements the v0.1 signalforge.manifest library surface for reading dbt manifest.json (v9–v12) and resolving a single model, alongside a full fixture corpus and research snapshot that supports the design decisions.
Changes:
- Add
signalforge.manifestsubpackage (Pydantic v2 models, loader, resolver, typed error hierarchy, public re-exports). - Add comprehensive unit/integration/error tests plus dbt fixture projects (small + medium) and negative fixtures for regression coverage.
- Vendor dbt research docs and add operational + contributor documentation for fixture regeneration and module usage.
Reviewed changes
Copilot reviewed 92 out of 103 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/signalforge/manifest/init.py | Defines the manifest module public API via re-exports and __all__. |
| src/signalforge/manifest/errors.py | Adds typed exceptions with remediation strings rendered in __str__. |
| src/signalforge/manifest/loader.py | Implements manifest I/O, version detection, symlink-hardened path resolution, and model resolver helpers. |
| src/signalforge/manifest/models.py | Adds frozen Pydantic v2 models for manifest nodes and thin Manifest method wrappers. |
| tests/manifest/test_public_api.py | Contract tests for exported surface and prevention of leaking internals. |
| tests/manifest/test_models.py | Unit + fixture validation tests for Pydantic models and drift detection. |
| tests/manifest/test_loader.py | Unit/integration/error tests for loader, resolver semantics, and symlink hardening. |
| tests/manifest/test_errors.py | Unit tests for error hierarchy, remediation defaults, and rendering behavior. |
| tests/fixtures/.gitignore | Ignores dbt-generated local artifacts to keep committed fixtures deterministic. |
| tests/fixtures/README.md | Documents fixture layout, regeneration procedures, and timestamp-scrubbing convention. |
| tests/fixtures/_generate_medium.py | Generates the synthetic medium dbt project SQL for reproducible fixtures. |
| tests/fixtures/regenerate.sh | Regenerates small-project manifests across schema versions via ephemeral dbt installs. |
| tests/fixtures/dbt_project_small/dbt_project.yml | Defines the small dbt project used for fixture manifests. |
| tests/fixtures/dbt_project_small/profiles.yml | DuckDB profile to regenerate fixtures without a real warehouse. |
| tests/fixtures/dbt_project_small/models/_sources.yml | Source definitions for the small dbt project fixtures. |
| tests/fixtures/dbt_project_small/models/staging/stg_users.sql | Small-project staging model fixture (source edge). |
| tests/fixtures/dbt_project_small/models/staging/stg_orders.sql | Disabled staging model fixture (disabled-dict coverage). |
| tests/fixtures/dbt_project_small/models/marts/dim_users.sql | Small-project mart model fixture (ref edge). |
| tests/fixtures/dbt_project_small/models/marts/fct_orders.sql | Small-project mart model fixture (ref chain). |
| tests/fixtures/dbt_project_small/target/manifest_v9.json | Committed v9 manifest fixture (dbt-core 1.5.x). |
| tests/fixtures/dbt_project_small/target/manifest_v10.json | Committed v10 manifest fixture (dbt-core 1.6.x). |
| tests/fixtures/dbt_project_small/target/manifest_v11.json | Committed v11 manifest fixture (dbt-core 1.7.x). |
| tests/fixtures/dbt_project_small/target/manifest_v12.json | Committed v12 manifest fixture (dbt-core 1.8.x). |
| tests/fixtures/dbt_project_medium/dbt_project.yml | Defines the medium dbt project used for scale fixtures. |
| tests/fixtures/dbt_project_medium/profiles.yml | DuckDB profile for the medium dbt project fixture regen. |
| tests/fixtures/dbt_project_medium/models/_sources.yml | Source definitions for the medium dbt project fixtures. |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_000.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_001.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_002.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_003.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_004.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_005.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_006.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_007.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_008.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_009.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_010.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_011.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_012.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_013.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_014.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_015.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_016.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_017.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_018.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_019.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_020.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_021.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_022.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_023.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_024.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_025.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_026.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_027.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_028.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_029.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_030.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_031.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_032.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_033.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_034.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_035.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_036.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_037.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_038.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_039.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_040.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_041.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_042.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_043.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_044.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_045.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_046.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_047.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_048.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/staging/stg_event_049.sql | Medium-project staging model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/marts/fct_events_000.sql | Medium-project mart model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/marts/fct_events_001.sql | Medium-project mart model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/marts/fct_events_002.sql | Medium-project mart model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/marts/fct_events_003.sql | Medium-project mart model fixture (generated). |
| tests/fixtures/dbt_project_medium/models/marts/fct_events_004.sql | Medium-project mart model fixture (generated). |
| tests/fixtures/dbt_project_medium/target/manifest_v12.json | Committed medium manifest fixture for scale/perf realism. |
| tests/fixtures/error_paths/malformed.json | Negative fixture for malformed JSON handling. |
| tests/fixtures/error_paths/missing_version_url.json | Negative fixture for missing schema URL + feature sniff. |
| tests/fixtures/error_paths/unsupported_v99.json | Negative fixture for unsupported schema version handling. |
| tests/fixtures/error_paths/disabled_only.json | Negative fixture where model exists only in disabled. |
| tests/fixtures/error_paths/empty_raw_code.json | Negative fixture where a model’s raw_code is empty. |
| docs/manifest-loader-ops.md | Operational guide for memory sizing, warnings, fixtures, and errors. |
| docs/research/README.md | Explains vendored research snapshot and refresh procedure. |
| docs/research/dbt-tooling-opportunity-report.md | Research synthesis on dbt tooling opportunities (snapshot). |
| docs/research/dbt-research-index.md | Research index and recommendation (snapshot). |
| docs/research/dbt-pain-deep-dive.md | Voice-of-user research and failure stories (snapshot). |
| docs/research/dbt-claude-technical-surface.md | Implementer reference for dbt artifacts + MCP + CI (snapshot). |
| docs/research/dbt-ai-tools-deep-dive.md | Survey of dbt AI tooling ecosystem (snapshot). |
| docs/research/dbt-tool-design-sketches.md | Tool design sketches and roadmap framing (snapshot). |
| docs/research/dbt-research.pdf | Compiled research PDF snapshot. |
| pyproject.toml | Adds runtime dep (Pydantic v2), dbt-core dev dep, and declares pytest markers. |
| README.md | Updates status/quick-start messaging to reflect manifest library landing first. |
| CONTRIBUTING.md | Documents pytest markers and fixture regeneration references. |
| CLAUDE.md | Updates repo status and documents the v0.1 manifest public API surface. |
| .claude/rules/testing-signal.md | Adds fixture-regeneration and drift-detector testing conventions. |
| .claude/rules/manifest-readers.md | Adds repository rule-of-thumb for external-format readers (Pydantic + path hardening). |
| plans/super/2-manifest-loader.md | Adds/updates the “super plan” for issue #2 with DEC decisions and story breakdown. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- .claude/rules/manifest-readers.md:33: example used PathOutsideProjectError (doesn't exist); align with the real ModelPathOutsideProjectError so contributors copy/pasting the rule into a future external-format reader pick up the correct symbol. - src/signalforge/manifest/loader.py:35-41: module docstring claimed get_model caches a "unique_id -> Model" index. The actual implementation caches only path-based indexes (manifest.nodes is the unique_id lookup). Docstring updated to match. - plans/super/2-manifest-loader.md:7,521: replace absolute maintainer worktree paths (/home/wesd/...) with portable placeholders / generic prose so the doc isn't workstation-specific. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wjduenow
added a commit
that referenced
this pull request
Apr 29, 2026
* Add super plan for #2 (manifest loader) and vendored dbt research - plans/super/2-manifest-loader.md: full /super-plan output (Phases 1-4) with 17 decisions and 9 stories (US-001..US-009 incl. Quality Gate and Patterns & Memory). - docs/research/: pinned snapshots of the seven dbt-prefixed research files from clauditor's gitignored docs/temp/, so contributors outside the maintainer's machine can read the references the plan cites (DEC-006). No code yet -- this is the planning PR. Implementation lands per the beads task graph after approval. * Fill in Beads Manifest for #2 plan (Phase 7 devolve) Epic bd_1-scaffolding-28p + 9 tasks created with 10 dependency edges matching the story graph. bd ready returns only .1 (US-001). * bd_1-scaffolding-28p.1: US-001 — add Pydantic v2 + dbt-core deps and pytest markers - pydantic>=2.5,<3 as the first runtime dep (DEC-001). - dbt-core>=1.8,<2 in [dev] for fixture regeneration (DEC-009). - pytest markers unit/integration/error declared (DEC-015). * bd_1-scaffolding-28p.3: US-003 — manifest errors module (TDD) - src/signalforge/manifest/{__init__.py, errors.py}: seven-class hierarchy rooted at ManifestError, with remediation: str rendered in __str__ (DEC-013, DEC-014). - tests/manifest/test_errors.py: ≥6 fail-capable unit tests covering hierarchy, default remediation, override, and base-class catching. * bd_1-scaffolding-28p.2: US-002 — test fixtures (small × 4 schemas, medium, 5 error paths) - tests/fixtures/dbt_project_small/: 4-5 model dbt project + manifest_v9/10/11/12.json (DEC-012). - tests/fixtures/dbt_project_medium/: ~50-model synthesised project + manifest_v12.json. - tests/fixtures/error_paths/: malformed/missing_version_url/unsupported_v99/disabled_only/empty_raw_code. - tests/fixtures/regenerate.sh + README.md (DEC-009 multi-version regen recipe). * bd_1-scaffolding-28p.4: US-004 — manifest Pydantic models module (TDD) - src/signalforge/manifest/models.py: Manifest, Model, Column, Ref, Config, DependsOn — frozen + extra=ignore + populate_by_name. Validators on unique_id (must start "model.") and raw_code (strip-to-None) per DEC-016. columns_list property on Model. Nested Config per DEC-011. - tests/manifest/test_models.py: TDD tests parametrised across the four committed v9/v10/v11/v12 small manifests + extra="forbid" drift detector. * bd_1-scaffolding-28p.5: US-005 — manifest loader module (TDD) - src/signalforge/manifest/loader.py: load(), _detect_version, _canonicalise_path with .resolve()+is_relative_to() symlink hardening (DEC-007), MAX_MANIFEST_BYTES soft warning (DEC-008), manifest_path override (DEC-010), get_model/iter_models/schema_version free functions backed by lazily-built unique_id and file-path indexes. - src/signalforge/manifest/models.py: thin method wrappers on Manifest delegating to loader free functions (deferred import, no cycle). - tests/manifest/test_loader.py: >=16 fail-capable tests including the 7 Phase 2 regression tests, symlink hardening, 200MB warning, all error paths. * bd_1-scaffolding-28p.6: US-006 — public __init__.py re-exports - src/signalforge/manifest/__init__.py: re-exports load, Manifest, Model, and all error classes; declares __all__ (DEC-017). - tests/manifest/test_public_api.py: programmatic importability check, Pydantic isinstance checks, error-hierarchy preservation, and explicit guards against accidental promotion of internals. * bd_1-scaffolding-28p.7: US-007 — documentation (research index, ops guide, CONTRIBUTING) - docs/research/README.md: snapshot framing for the seven vendored dbt research files (DEC-006). - docs/manifest-loader-ops.md: memory profile table, soft 200MB warning posture (DEC-008), multi-version regen cross-link (DEC-009/DEC-012), supported-version table, error-class quick reference. - CONTRIBUTING.md: Test markers + Regenerating fixtures subsections (DEC-015 / DEC-009). - README.md: v0.1 status callout updated to reflect library-first delivery. * bd_1-scaffolding-28p.8: Quality gate — fix bugs from code-reviewer passes 1-4 Pass 2 (adversarial) surfaced two real bugs in the path-hardening logic; both are now fixed with regression tests that fail before the fix. 1. Default target/manifest.json bypassed _canonicalise_path. A symlink at target/manifest.json -> /etc/passwd would have been silently followed. Fix: route the default path through _canonicalise_path the same way the manifest_path override is handled. (DEC-007 hardening, applies uniformly.) 2. Path.resolve() raises RuntimeError on symlink cycles regardless of strict=. _canonicalise_path leaked that bare RuntimeError instead of the typed ModelPathOutsideProjectError our hierarchy promises. Fix: wrap both resolve() calls in try/except RuntimeError and re-raise with the same remediation surface. Three new fail-capable regression tests: - test_default_manifest_path_symlink_escape_is_rejected - test_symlink_loop_in_default_path_is_rejected - test_symlink_loop_in_explicit_manifest_path_is_rejected Pass 3 (test correctness) flagged one marker mis-mark: test_oversize_manifest_emits_warning was @pytest.mark.unit but reads a real fixture file; corrected to @pytest.mark.integration. Validation: ruff/format/pyright all green; pytest 67 passed (64 → 67, +3 regression tests). * bd_1-scaffolding-28p.9: US-009 — patterns & memory - .claude/rules/manifest-readers.md (new): Pydantic v2 frozen+extra=ignore for external-format readers; the three symlink-resolution traps caught by issue #2's pass-2 review (Path.relative_to vs resolve, RuntimeError on cycles, default paths must be canonicalised too); ManifestError + remediation: str pattern; no logging/metrics in stage-0 modules. - .claude/rules/testing-signal.md: appended sections on fixture regeneration via ephemeral uvx and the extra="forbid" drift detector pattern (DEC-005, DEC-009, DEC-012, DEC-017). - CLAUDE.md: marked issue #2 shipped; added "Public API surface (v0.1)" section listing signalforge.manifest as the first stable surface; cross-link to docs/manifest-loader-ops.md. Auto-memory entry added (outside the repo): the editable-install race that bit US-002's worker-in-worktree pattern — orchestrators must reinstall from the merged feature worktree before validating. * Address Copilot review comments on PR #15 - .claude/rules/manifest-readers.md:33: example used PathOutsideProjectError (doesn't exist); align with the real ModelPathOutsideProjectError so contributors copy/pasting the rule into a future external-format reader pick up the correct symbol. - src/signalforge/manifest/loader.py:35-41: module docstring claimed get_model caches a "unique_id -> Model" index. The actual implementation caches only path-based indexes (manifest.nodes is the unique_id lookup). Docstring updated to match. - plans/super/2-manifest-loader.md:7,521: replace absolute maintainer worktree paths (/home/wesd/...) with portable placeholders / generic prose so the doc isn't workstation-specific. ---------
wjduenow
added a commit
that referenced
this pull request
May 11, 2026
* Initial commit
* Add initial README describing SignalForge
Pre-alpha design document. Establishes the value proposition (LLM-drafted
dbt artifacts pruned against real warehouse data), the differentiator
(quality eval in the loop, reusing clauditor's grading methodology), the
v0.1-v1.0 roadmap, and the design principles.
* Switch v0.1 target warehouse from Snowflake to BigQuery
Architecture stays warehouse-agnostic; adapters plug in behind a thin
sampling/profiling interface. BigQuery picked as v0.1 target for its
generous sampled-read pricing and INFORMATION_SCHEMA.JOBS history.
Snowflake, Databricks, Postgres, Redshift moved to v0.2+.
* Add CLAUDE.md, .gitignore, maintainer skills, and #1 plan doc
Sets up per-repo policy for AI tooling: synced commands/agents and
non-maintainer skills stay local-only, only release-manager and
review-agentskills-spec ship with the repo. Pattern mirrors clauditor.
Includes the super-plan document for issue #1 (project scaffolding) so
planning work survives across worktrees.
* 1: Project scaffolding (plan) (#14)
* Trim bark/beads scaffolding additions to match repo policy
- CLAUDE.md: replace mandatory beads block with a short
availability note. The original asserted bd as the only allowed
task tracker and prescribed a fixed end-of-session push workflow,
which conflicts with /super-plan and Claude Code's task tools.
- AGENTS.md: same trim; keep the non-interactive shell guidance.
- .claude/settings.json: track. Just bd-prime hooks for SessionStart
and PreCompact — no user-specific paths.
- .gitignore: ignore .claude/plugins/ (install/cache state).
* Plan #1: Phases 2-3 complete (architecture review + decisions)
- Architecture review surfaces 8 concerns, 0 blockers (CI permissions,
action SHA-pinning, smoke test signal, hatchling wheel target,
CONTRIBUTING scope, README drift, version value).
- R1-R6 resolved as DEC-009..DEC-014.
- Filed #13 to track the open question of long-term beads ↔
/super-plan integration; link recorded in the plan doc.
- Add .beads/ to .gitignore — bark's bd init landed it in the
original checkout, not the worktree; ignoring prevents accidental
commits if it appears in a future checkout too.
* Plan #1: Phase 4 — story breakdown (US-001..US-007)
Seven stories: five implementation (foundation, lint+type, tests,
CI, docs), one quality gate, one patterns-and-memory. Each story
traces to specific DEC-### entries and lists explicit files,
acceptance criteria, and dependencies.
Validation command set: pip install -e .[dev] && ruff check . &&
pyright && pytest.
* Plan #1: phase published (PR #14)
* Plan #1: phase devolved (bd epic bd_1-scaffolding-mxk + 7 tasks)
Created the bd issue graph: 1 epic, 7 tasks (US-001..US-007), 11
blocks edges. bd ready returns only US-001 — the rest are correctly
blocked.
Discovered (and recorded) that bd is worktree-aware via 'bd context':
auto-discovers the canonical .beads/ at the main checkout from any
worktree CWD. No symlink or BD_REPO_DIR env var needed; obsoletes
the option-C plan.
* bd_1-scaffolding-mxk.1: Add pyproject.toml + src layout + __version__
Hatchling backend with dynamic version sourced from
src/signalforge/__init__.py. Explicit wheel packages declaration
(DEC-011) since hatchling does not reliably auto-find src layout.
Dev deps: ruff, pyright, pytest (DEC-004).
Closes US-001 of #1.
* bd_1-scaffolding-mxk.2: Add ruff + pyright config
Per DEC-002: line-length 100, target-version py310, rule set
E/F/W/I/UP/B/SIM, pyright in standard mode with src+tests roots
and Python 3.11 (matches CI). Empty [tool.ruff.format] accepts
defaults.
Closes US-002 of #1.
* bd_1-scaffolding-mxk.5: Add CONTRIBUTING.md + README v0.1 status callout
Lean CONTRIBUTING (DEC-012): branch policy, local dev commands,
license reminder, explicit out-of-scope note for bark/super-plan/bd
(tracked in #13). README gets a sharper v0.1 status block above the
quick-start (DEC-013) so readers do not try a 'pip install signalforge'
that fails today.
Closes US-005 of #1.
* bd_1-scaffolding-mxk.3: Add tests/ + smoke test + pytest config
Smoke test imports signalforge and asserts __version__ matches
PEP 440 shape (DEC-010 — no assert True; the test must be capable
of failing). pytest config sets testpaths=['tests'] and
addopts='-ra --strict-markers' so unknown markers error out
loudly instead of being silently accepted.
Closes US-003 of #1.
* bd_1-scaffolding-mxk.4: Add GitHub Actions CI workflow
DEC-009: actions/checkout and actions/setup-python pinned to commit
SHA with trailing version comment. Top-level permissions scoped to
contents:read. Concurrency cancels superseded runs on the same ref.
DEC-003: Python pinned to 3.11 (single version for v0.1; widen later).
Triggers: PRs into dev, pushes to main. Job runs ruff check, ruff
format check, pyright, pytest in series — same commands contributors
run locally per CONTRIBUTING.md.
Closes US-004 of #1.
* bd_1-scaffolding-mxk.6: Quality gate — fix bugs from code review
Quote pip install extras for zsh users (CONTRIBUTING.md, README.md):
'pip install -e .[dev]' is a glob in zsh and fails with 'no matches
found'. Quoting works in both bash and zsh. Caught by QG pass 3.
Fix README v0.1 prose drift: line 49 still said 'Snowflake adapter'
after commit 243a709 switched v0.1 to BigQuery, contradicting the
roadmap table on the next line. Caught by QG pass 4.
Closes US-006 of #1.
* bd_1-scaffolding-mxk.7: Add .claude/rules/* and CLAUDE.md validation cmd
Capture the conventions established by issue #1 so future
/super-plan runs in this repo discover them automatically:
- python-build.md: Hatchling + src layout + dynamic version (DEC-011);
the "wheel target packages declaration is non-negotiable" lesson.
- ci-supply-chain.md: SHA-pinning, scoped permissions, concurrency
cancel-in-progress (DEC-003, DEC-009).
- testing-signal.md: no assert-True tests; pytest 9 needs both
addopts=--strict-markers AND strict_markers=true ini setting,
because the option does not propagate to getini (DEC-010, US-003).
CLAUDE.md gains a 'Validation' section with the canonical four-step
command shared between local dev and CI; 'Repository status' now
reflects what shipped instead of "only README and LICENSE".
Implemented by orchestrator after the worker hit a permission gap on
.claude/ writes inside the worktree (filed for the next ralph run to
include .claude/** in worker scope).
Closes US-007 of #1.
* Address Copilot PR #14 review comments
- testing-signal.md: replace "(CLAUDE.md §1)" with explicit reference to
the "Architectural commitments" heading (CLAUDE.md has no numbered §s).
- test_smoke.py: rewrite the misleading comment — the test does not
exercise wheel packaging (editable install goes via .pth, not the
wheel target). Describe what is actually verified: idempotent import.
- .github/workflows/ci.yml: quote ".[dev]" to match the documented
zsh-safe convention even though Actions runs bash.
- README.md: "follow-up ticket of v0.1" → "follow-up v0.1 ticket".
- AGENTS.md: drop `bd dolt push` from the quick reference and replace
with a comment pointing to #13 (the dolt remote URL is not wired up
in this repo, so the command would fail).
- plans/super/1-project-scaffolding.md:
- Replace local absolute paths in Meta, Phase 2 housekeeping note,
and Beads Manifest with machine-agnostic descriptions.
- Update Phase from "devolved" to "implemented" (all 7 stories
merged; PR ready for review).
- Align the Detailed Breakdown's stated validation command with the
canonical CLAUDE.md §Validation recipe (quoted extras + ruff
format check).
- Fix US-003 acceptance criteria: 3 tests with their actual names
(was "1 test"; old name placeholder was wrong); add the pytest 9
strict-markers ini-vs-addopts caveat inline.
Validation green: ruff check, ruff format --check, pyright, pytest
(3 passed).
---------
* 2: dbt manifest loader (plan) (#15)
* Add super plan for #2 (manifest loader) and vendored dbt research
- plans/super/2-manifest-loader.md: full /super-plan output (Phases 1-4)
with 17 decisions and 9 stories (US-001..US-009 incl. Quality Gate
and Patterns & Memory).
- docs/research/: pinned snapshots of the seven dbt-prefixed research
files from clauditor's gitignored docs/temp/, so contributors outside
the maintainer's machine can read the references the plan cites
(DEC-006).
No code yet -- this is the planning PR. Implementation lands per the
beads task graph after approval.
* Fill in Beads Manifest for #2 plan (Phase 7 devolve)
Epic bd_1-scaffolding-28p + 9 tasks created with 10 dependency edges
matching the story graph. bd ready returns only .1 (US-001).
* bd_1-scaffolding-28p.1: US-001 — add Pydantic v2 + dbt-core deps and pytest markers
- pydantic>=2.5,<3 as the first runtime dep (DEC-001).
- dbt-core>=1.8,<2 in [dev] for fixture regeneration (DEC-009).
- pytest markers unit/integration/error declared (DEC-015).
* bd_1-scaffolding-28p.3: US-003 — manifest errors module (TDD)
- src/signalforge/manifest/{__init__.py, errors.py}: seven-class hierarchy
rooted at ManifestError, with remediation: str rendered in __str__
(DEC-013, DEC-014).
- tests/manifest/test_errors.py: ≥6 fail-capable unit tests covering
hierarchy, default remediation, override, and base-class catching.
* bd_1-scaffolding-28p.2: US-002 — test fixtures (small × 4 schemas, medium, 5 error paths)
- tests/fixtures/dbt_project_small/: 4-5 model dbt project + manifest_v9/10/11/12.json (DEC-012).
- tests/fixtures/dbt_project_medium/: ~50-model synthesised project + manifest_v12.json.
- tests/fixtures/error_paths/: malformed/missing_version_url/unsupported_v99/disabled_only/empty_raw_code.
- tests/fixtures/regenerate.sh + README.md (DEC-009 multi-version regen recipe).
* bd_1-scaffolding-28p.4: US-004 — manifest Pydantic models module (TDD)
- src/signalforge/manifest/models.py: Manifest, Model, Column, Ref, Config,
DependsOn — frozen + extra=ignore + populate_by_name. Validators on
unique_id (must start "model.") and raw_code (strip-to-None) per DEC-016.
columns_list property on Model. Nested Config per DEC-011.
- tests/manifest/test_models.py: TDD tests parametrised across the four
committed v9/v10/v11/v12 small manifests + extra="forbid" drift detector.
* bd_1-scaffolding-28p.5: US-005 — manifest loader module (TDD)
- src/signalforge/manifest/loader.py: load(), _detect_version,
_canonicalise_path with .resolve()+is_relative_to() symlink hardening
(DEC-007), MAX_MANIFEST_BYTES soft warning (DEC-008), manifest_path
override (DEC-010), get_model/iter_models/schema_version free functions
backed by lazily-built unique_id and file-path indexes.
- src/signalforge/manifest/models.py: thin method wrappers on Manifest
delegating to loader free functions (deferred import, no cycle).
- tests/manifest/test_loader.py: >=16 fail-capable tests including the 7
Phase 2 regression tests, symlink hardening, 200MB warning, all error
paths.
* bd_1-scaffolding-28p.6: US-006 — public __init__.py re-exports
- src/signalforge/manifest/__init__.py: re-exports load, Manifest, Model,
and all error classes; declares __all__ (DEC-017).
- tests/manifest/test_public_api.py: programmatic importability check,
Pydantic isinstance checks, error-hierarchy preservation, and
explicit guards against accidental promotion of internals.
* bd_1-scaffolding-28p.7: US-007 — documentation (research index, ops guide, CONTRIBUTING)
- docs/research/README.md: snapshot framing for the seven vendored dbt
research files (DEC-006).
- docs/manifest-loader-ops.md: memory profile table, soft 200MB warning
posture (DEC-008), multi-version regen cross-link (DEC-009/DEC-012),
supported-version table, error-class quick reference.
- CONTRIBUTING.md: Test markers + Regenerating fixtures subsections
(DEC-015 / DEC-009).
- README.md: v0.1 status callout updated to reflect library-first delivery.
* bd_1-scaffolding-28p.8: Quality gate — fix bugs from code-reviewer passes 1-4
Pass 2 (adversarial) surfaced two real bugs in the path-hardening logic;
both are now fixed with regression tests that fail before the fix.
1. Default target/manifest.json bypassed _canonicalise_path. A symlink at
target/manifest.json -> /etc/passwd would have been silently followed.
Fix: route the default path through _canonicalise_path the same way
the manifest_path override is handled. (DEC-007 hardening, applies
uniformly.)
2. Path.resolve() raises RuntimeError on symlink cycles regardless of
strict=. _canonicalise_path leaked that bare RuntimeError instead of
the typed ModelPathOutsideProjectError our hierarchy promises. Fix:
wrap both resolve() calls in try/except RuntimeError and re-raise
with the same remediation surface.
Three new fail-capable regression tests:
- test_default_manifest_path_symlink_escape_is_rejected
- test_symlink_loop_in_default_path_is_rejected
- test_symlink_loop_in_explicit_manifest_path_is_rejected
Pass 3 (test correctness) flagged one marker mis-mark:
test_oversize_manifest_emits_warning was @pytest.mark.unit but reads a
real fixture file; corrected to @pytest.mark.integration.
Validation: ruff/format/pyright all green; pytest 67 passed
(64 → 67, +3 regression tests).
* bd_1-scaffolding-28p.9: US-009 — patterns & memory
- .claude/rules/manifest-readers.md (new): Pydantic v2 frozen+extra=ignore
for external-format readers; the three symlink-resolution traps caught
by issue #2's pass-2 review (Path.relative_to vs resolve, RuntimeError
on cycles, default paths must be canonicalised too); ManifestError +
remediation: str pattern; no logging/metrics in stage-0 modules.
- .claude/rules/testing-signal.md: appended sections on fixture
regeneration via ephemeral uvx and the extra="forbid" drift detector
pattern (DEC-005, DEC-009, DEC-012, DEC-017).
- CLAUDE.md: marked issue #2 shipped; added "Public API surface (v0.1)"
section listing signalforge.manifest as the first stable surface;
cross-link to docs/manifest-loader-ops.md.
Auto-memory entry added (outside the repo): the editable-install race
that bit US-002's worker-in-worktree pattern — orchestrators must
reinstall from the merged feature worktree before validating.
* Address Copilot review comments on PR #15
- .claude/rules/manifest-readers.md:33: example used PathOutsideProjectError
(doesn't exist); align with the real ModelPathOutsideProjectError so
contributors copy/pasting the rule into a future external-format reader
pick up the correct symbol.
- src/signalforge/manifest/loader.py:35-41: module docstring claimed
get_model caches a "unique_id -> Model" index. The actual implementation
caches only path-based indexes (manifest.nodes is the unique_id lookup).
Docstring updated to match.
- plans/super/2-manifest-loader.md:7,521: replace absolute maintainer
worktree paths (/home/wesd/...) with portable placeholders / generic
prose so the doc isn't workstation-specific.
---------
* 3: BigQuery warehouse adapter (plan) (#17)
* 3: BigQuery warehouse adapter (plan)
Super plan for issue #3 — BigQuery warehouse adapter with sampling +
dialect helpers. Covers Discovery + Architecture Review + Refinement
Log + Detailed Breakdown.
Phase 1 — DEC-001..012 lock the subpackage layout, ABC shape, sampling
strategy, and ops surface.
Phase 2 — six parallel reviews (security/performance/data-model/API/
observability/testing) surfaced 9 blockers + 18 concerns. No findings
invalidate the Phase-1 shape.
Phase 3 — DEC-013..028 resolve every blocker and concern. Notable:
PartitionFilter ADT replaces raw SQL strings (B2); use_query_cache=False
to preserve determinism (B3); DbtProfileTarget extra="forbid" + auth
validator to make ADC fallback loud (B5); WarehouseAdapter.from_profile
factory (B7); TestResult.explanation() to anchor explainable diffs (B8).
Phase 4 — 14 stories, architecture-ordered. ~80 unit tests
enumerated; 6 integration tests gated by SF_RUN_BQ.
* 3: devolve plan to beads (epic + 14 tasks)
Phase 5 → 7. Epic bd_1-scaffolding-8xk with 14 typed task children:
US-001..US-012 implementation, US-013 Quality Gate, US-014 Patterns &
Memory. Dependency graph wired so only US-001 is initially ready;
predecessors unlock as they close.
Plan doc Beads Manifest section now lists every bead ID and its deps.
Phase: published → devolved.
* bd_1-scaffolding-8xk.1: US-001 — Wire BigQuery deps + pytest bigquery marker
Add google-cloud-bigquery and PyYAML runtime deps, types-PyYAML dev dep,
register the `bigquery` pytest marker, and gate it from default collection
via `-m 'not bigquery'`.
Refs DEC-010, DEC-021 in plans/super/3-bigquery-adapter.md.
* bd_1-scaffolding-8xk.2: US-002 — dbt profile YAML fixtures
Hand-author six dbt profile fixtures under tests/fixtures/profiles/ for
US-005's profile-loader tests:
- bigquery_oauth.yml — minimal valid ADC profile (the v0.1 happy path).
- bigquery_service_account.yml — non-oauth method; drives
UnsupportedAuthMethodError (DEC-017).
- multi_target.yml — two outputs; drives the target= override test.
- missing_target.yml — target: dev with only prod defined; drives
ProfileTargetNotFoundError.
- dbt_project.yml — minimal project file for project-root resolution.
- dbt_bigquery_drift_v1_9.yml — every documented dbt-bigquery 1.9 oauth
field; drives the extra="forbid" strict-model drift detector.
Mirrors the dbt-bigquery 1.9 docs and is bumped manually when dbt-bigquery
releases a new minor — no regeneration script (DEC-009, DEC-017).
Update tests/fixtures/README.md with a new "Profiles" section explaining
the regeneration trigger, the source URL, and the hand-author rationale.
All six files load cleanly via yaml.safe_load. Validation passes: ruff,
ruff format, pyright (0/0/0), pytest (67 passed).
* bd_1-scaffolding-8xk.3: US-003 — Warehouse errors module (DEC-026, DEC-022)
Implements `signalforge.warehouse.errors`: a 16-class typed exception
hierarchy rooted at `WarehouseError`, mirroring the manifest layer's
remediation pattern. Every error carries a class-level
`default_remediation` that the base `__str__` renders on a separate
`↳ Remediation:` line — the explainable-diffs commitment, applied at
the warehouse-adapter failure surface.
DEC-022: user-supplied strings (table names, identifiers, profile
fields, paths) route through a private `_format_value` helper that
quotes via `repr()` so adversarial input cannot smuggle special
characters into log viewers / stack traces.
DEC-026: 15 typed subclasses listed in the plan are implemented;
`ProfileTargetNotFoundError` inherits from `ProfileNotFoundError`,
`SamplingRequiresPartitionFilterError` and `UnknownTableSizeError`
inherit from `SamplingError`, so callers can catch with the parent
when they don't care which it is.
Also:
- `signalforge.warehouse` package skeleton (`__init__.py` near-empty;
US-011 will wire up the full re-export surface).
- `pyproject.toml`: pytest `--import-mode=importlib` so
`tests/manifest/test_errors.py` and `tests/warehouse/test_errors.py`
can share a basename without adding `tests/__init__.py` (preserves
the no-init rule from `testing-signal.md`).
- 15 unit tests covering remediation rendering, repr-quoting of
adversarial input, parent-catches-child semantics for the two
inheritance chains, field accessibility on `BytesBilledExceededError`,
and a `_CONSTRUCT_KWARGS` table that keeps every class in `__all__`
exercised.
Validation: ruff check + ruff format --check + pyright + pytest all
pass (82 tests).
* bd_1-scaffolding-8xk.4: US-004 — Warehouse models (DEC-003, 004, 013, 014, 016, 018, 020, 027)
Implements the five typed return types for the warehouse adapter
layer plus two private helper modules:
- src/signalforge/warehouse/models.py — Dialect (+BIGQUERY_DIALECT
constant), TableRef (+from_model gateway), PartitionFilter,
ColumnStats, TestResult (with explanation()).
- src/signalforge/warehouse/_sql_safety.py — DEC-013 identifier
regex helper and run_test_sql validator.
- src/signalforge/warehouse/_test_result_repr.py — DEC-020
type-aware compact row rendering used by TestResult.explanation.
- tests/warehouse/test_models.py — 16 tests covering identifier
validation, alias-over-name resolution, complex-type min/max,
per-op PartitionFilter construction, and the explanation
rendering surface.
Mirrors signalforge.manifest.models conventions (Pydantic v2,
frozen=True). TableRef/Dialect/PartitionFilter are frozen
dataclasses (constructed by SignalForge code, not deserialised);
ColumnStats/TestResult are Pydantic models so they can round-trip
through the future JSON cache.
TestResult.__test__ = False suppresses a pytest collection warning
(class name starts with "Test" but it is a data class).
Validation: ruff + ruff format + pyright + pytest all green;
98 tests pass.
* bd_1-scaffolding-8xk.7: US-007 — FakeBigQueryClient (DEC-002, DEC-028)
Hand-rolled fake for google.cloud.bigquery.Client with an explicit
expect_query / expect_get_table / expect_list_rows API. Unexpected calls
raise AssertionError; assert_all_expectations_met() flags unconsumed
expectations. Lives in tests/warehouse/ — never imported by production
code.
Avoids two anti-patterns: pytest-bigquery-mock (dead, targets bq 2.x) and
unittest.mock.MagicMock (auto-passes everything → always-pass tests, in
violation of testing-signal.md).
Adds six self-tests so regressions in the fake itself don't masquerade
as adapter bugs upstream. Test count rises 98 → 104.
* bd_1-scaffolding-8xk.5: US-005 — Profiles loader module (DEC-009, 017, 022, 023)
Add `signalforge.warehouse.profiles` with `DbtProfileTarget` (Pydantic v2
frozen, `extra="forbid"`, `populate_by_name=True`, `method` field validator
that raises `UnsupportedAuthMethodError` for anything other than `oauth`/None)
and `load_profile(project_dir, target=None) -> DbtProfileTarget`.
Resolution order per DEC-009: `$DBT_PROFILES_DIR/profiles.yml` (user-trusted)
→ `<project_dir>/profiles.yml` (symlink-hardened via `_path_safety`) →
`~/.dbt/profiles.yml` (user-trusted). Active target = arg → profile's
`target:` field → `ProfileTargetNotFoundError`. Profile name read from
`<project_dir>/dbt_project.yml`'s `profile:` field. Soft 1 MB warning logged
to `signalforge.warehouse` (DEC-023, threshold patchable from tests).
`_path_safety.canonicalise_path` mirrors the manifest loader's
`_canonicalise_path` precedent — three traps from `manifest-readers.md`:
`.resolve()` before `.is_relative_to()`, wrap `RuntimeError` for symlink
cycles, gate the default path the same as user-supplied input. Per DEC-017
the helper is copied (not imported) to keep subpackages decoupled; promotion
to a shared utility is a US-014 follow-up.
Tests: 13 new tests covering all three resolution paths, target override,
missing target, missing-everywhere, unsupported method, unknown-field
strictness, `schema:` alias, symlink rejection, soft warning, drift detector
(test-only StrictModel mirroring every dbt-bigquery 1.9 oauth field), and
logger discipline. Test count 98 → 111. `yaml.safe_load` only.
* bd_1-scaffolding-8xk.6: US-006 — WarehouseAdapter ABC + factory
Add `signalforge.warehouse.base.WarehouseAdapter` (abstract sampler /
profiler / test-runner) plus the `from_profile` classmethod (DEC-019)
that lazy-imports `BigQueryAdapter` for `profile.type == "bigquery"`
and raises `UnsupportedProfileTypeError` otherwise. Concrete adapters
land under `signalforge.warehouse.adapters/<warehouse>.py`; `bigquery.py`
ships a skeleton — constructor + `__repr__` (DEC-022 credential redaction)
+ `dialect()` returning the live `BIGQUERY_DIALECT` constant — with the
remaining abstract methods raising `NotImplementedError` until US-008.
Tests cover ABC enforcement, factory dispatch, the unsupported-type
branch, and both halves of DEC-019's max_bytes_billed fallback contract.
Test count 117 → 122; ruff/format/pyright/pytest all green.
* bd_1-scaffolding-8xk.8: US-008 — BigQueryAdapter full implementation
Replace the US-006 stub with the full v0.1 BigQuery adapter:
* `_client.py` — duck-typed `_BQClientProtocol` (matches both
`bigquery.Client` and `FakeBigQueryClient`), `make_real_client`,
`make_query_job_config` (DEC-015 defaults), `map_bq_exception`
(`google.api_core` → typed `WarehouseError` translation), and
`row_to_dict`. All `# pyright: ignore[...]` noise is contained here.
* `bigquery.py` — full `BigQueryAdapter`:
- `sample_rows` per DEC-006/DEC-024: deterministic
`MOD(ABS(FARM_FINGERPRINT(TO_JSON_STRING(t))), bucket) < 1`
with `Table.num_rows`-driven bucket sizing and fail-loud
thresholds (`UnknownTableSizeError`,
`SamplingRequiresPartitionFilterError`).
- `column_stats` per DEC-008/DEC-025: requires active context,
accumulates per-table, batched flush on first read; complex BQ
types (`GEOGRAPHY`/`JSON`/`BYTES`/`ARRAY`/`STRUCT`/`RANGE`) skip
`MIN/MAX` per DEC-016; soft `WARNING` at >500-column batches
(DEC-023).
- `run_test_sql` per DEC-007: wraps in `COUNT(*) [+ ARRAY_AGG]`
aggregate; returns typed `TestResult` with `row_schema=None`
(v0.2 will dry_run for inner schema).
- DEC-013 identifier validation re-checked at `column_stats`
entry; DEC-014 partition rendering by Python type;
DEC-022-redacted `__repr__`; DEC-027 `project=None` resolves
to client.project at quote time.
* `tests/warehouse/test_bigquery_smoke.py` — 7 smoke tests pinning
`_quote` resolution, `_render_partition_filter` rendering for
date/datetime/str values (incl. `'`-escaping), the
outside-context `RuntimeError`, and `__repr__` redaction.
Comprehensive unit tests land in US-009.
129 tests pass (was 122). ruff/format/pyright all clean.
* bd_1-scaffolding-8xk.11: US-011 — finalise signalforge.warehouse public API
Replace the skeleton ``__all__: list[str] = []`` in
``signalforge/warehouse/__init__.py`` with a thin re-export layer
mirroring ``signalforge.manifest`` (DEC-017):
- 10 non-error names (``load_profile``, ``WarehouseAdapter``,
``BigQueryAdapter``, ``Dialect``, ``TableRef``, ``PartitionFilter``,
``ColumnStats``, ``TestResult``, ``DbtProfileTarget``, and the
``BIGQUERY_DIALECT`` constant)
- the full 16-class ``WarehouseError`` hierarchy from
``signalforge.warehouse.errors``
``__all__`` is a hard-coded sorted literal (not ``sorted([...])``) so
pyright's ``reportUnsupportedDunderAll`` stays happy; sort order is
guarded by a regression test instead.
Add ``tests/warehouse/test_public_api.py`` mirroring
``tests/manifest/test_public_api.py``: six contract tests covering
binding, no underscore leakage, sort order, helper isolation, the
required-name minimum, and full re-export of
``signalforge.warehouse.errors.__all__``.
Test count rises from 129 to 135. Validation: ruff check, ruff format
--check, pyright, pytest all clean.
* bd_1-scaffolding-8xk.10: US-010 — BigQuery integration tests (gated)
Add six maintainer-only integration tests under
`tests/warehouse/test_bigquery_integration.py` that exercise
`BigQueryAdapter` against `bigquery-public-data.samples.shakespeare`.
Each test wears both `@pytest.mark.bigquery` (filtered by the default
`addopts = -m 'not bigquery'`) and `@pytest.mark.skipif(not SF_RUN_BQ)`
(belt-and-suspenders per DEC-011/DEC-021): the default `pytest` run
deselects all six and the test count stays at 129; `pytest -m bigquery`
collects exactly the six new tests.
Coverage:
- `sample_rows` + `column_stats` round-trip on Shakespeare;
- `run_test_sql` clean (`WHERE FALSE`) and dirty (`LIMIT 5`,
`capture_failures=3`) paths;
- `BytesBilledExceededError` via `max_bytes_billed=1` on Shakespeare
(DEC-028 — free since BQ rejects the dry-run pre-flight);
- `WarehouseAuthError` via `monkeypatch` on `google.auth.default`
(DEC-028 — exercises the lazy-client construction path).
The Shakespeare `TableRef` is built inside the test bodies via
`__new__` + `object.__setattr__` to bypass DEC-013's strict
identifier regex (real BQ project IDs may contain hyphens; the
adapter's backtick-quoted `_quote` path handles them fine).
Loosening the project-id regex is tracked separately.
Add a "BigQuery integration tests" section to `CONTRIBUTING.md`
documenting the `gcloud auth application-default login` +
`SF_RUN_BQ=1 pytest -m bigquery` flow.
* bd_1-scaffolding-8xk.12: US-012 — warehouse adapter ops doc
Add docs/warehouse-adapter-ops.md per DEC-012 + DEC-027 covering quick
start (ADC + load_profile + from_profile), profile resolution, cost
defaults (max_bytes_billed, use_query_cache=False rationale, BQ job
labels), sampling strategy with the TABLESAMPLE cost-asterisk and
PartitionFilter use, the with-block contract for column_stats batching,
SF_RUN_BQ-gated integration tests, debugging via the
signalforge.warehouse logger and typed-error fields, and a full table
of all 16 typed exceptions in signalforge.warehouse.errors.
Cross-link from README's new Configuration section and from
docs/research/dbt-research-index.md back to the ops doc and the design
plan.
Validation: ruff/format/pyright/pytest all green; test count unchanged
at 129.
* bd_1-scaffolding-8xk.9: US-009 — comprehensive BigQueryAdapter unit tests
Adds tests/warehouse/conftest.py (centralised fake_client / adapter /
table_ref / shakespeare_table fixtures) and tests/warehouse/test_bigquery_unit.py
(45 tests covering cost defaults, _default_job_config DEC-015 plumbing,
dialect identity, sample_rows DEC-006/DEC-024 decision tree, column_stats
DEC-008/DEC-016/DEC-023 batching + complex-type MIN/MAX skipping,
run_test_sql DEC-007/DEC-013 wrapping + SQL-safety rejects, exception
mapping for BadRequest / NotFound / DefaultCredentials, and the
DEC-025 context-manager lifecycle).
Test count: 135 → 180 (+45). All four validation checks pass.
Notes on plan deviations:
* Skipped test_run_test_sql_populates_row_schema (v0.2 behaviour); replaced
with test_run_test_sql_row_schema_is_none asserting the v0.1 contract.
* Skipped test_per_call_max_bytes_caps_downward_only (v0.2).
* test_default_max_bytes_billed_via_from_profile is already covered in
test_base.py — not duplicated here.
* SQL-text introspection uses a query-wrapper helper (_wrap_query_capture)
rather than a custom regex matcher, since FakeBigQueryClient.expect_query
re-compiles `matching` through re.compile and rejects non-Pattern objects.
* test_column_stats_warns_at_threshold pre-seeds _column_stats_pending to
trip the threshold; the simplified Option A flushes after every call so
the pending list never grows past 1 via public API alone.
* bd_1-scaffolding-8xk.13: Quality Gate — fix 8 findings from code review pass 1
1. (BLOCKER) TableRef now accepts hyphenated GCP project IDs via a new
_PROJECT_RE / validate_project_id helper; strict identifier regex
still gates dataset / table / column. Integration test drops the
__new__ + object.__setattr__ bypass.
2. (MAJOR) _render_partition_filter escapes "\\" before "'" so an
adversarial trailing "\\'" cannot terminate the BQ string literal
early.
3. (MAJOR) Ops doc column_stats section now documents v0.1's eager-flush
semantics (one query per call) and points at the v0.2 lazy-proxy
follow-up. New "v0.2 follow-ups" section tracks both gaps.
4. (MAJOR) Ops doc cites the real signalforge_stage values
(warehouse_sample, warehouse_stats, warehouse_test) and lists them
under Cost defaults.
5. (MINOR) map_bq_exception accepts context={"max_bytes_billed": ...}
so BytesBilledExceededError renders the configured cap instead of 0.
6. (MINOR) sample_rows adds ORDER BY FARM_FINGERPRINT(TO_JSON_STRING(t))
before LIMIT so truncation is deterministic when the bucket WHERE
retains more than n rows (DEC-006).
7. (MINOR) NotFound "column" branch removed (dead in real BQ); missing
columns now route through a BadRequest sub-branch matching
"Unrecognized name" / "name not found".
8. (MINOR) make_query_job_config drops the # pragma: no cover; gains a
default version (resolved from signalforge.__version__) and a unit
test pinning use_query_cache=False, the maximum_bytes_billed value,
and the labels dict.
Validation: ruff check + ruff format --check + pyright + pytest all
green; 186 default tests pass (was 180; +6 new tests across findings 1,
2, 5, 6, 7, 8) with the 6 BigQuery-marked tests still deselected.
* bd_1-scaffolding-8xk.13: Quality Gate — fix 3 minor findings from review pass 2
- Drop legacy domain-scoped GCP project ID claim from _PROJECT_RE
docstring; the regex never accepted them and _quote() doesn't split
on ':'. Tracked as v0.2 follow-up.
- Bound the strict-identifier fallback in _PROJECT_RE to 6-30 chars to
match GCP's documented limits (was: any length).
- Escape '\\' and "'" in compact_repr's string-value branch so
TestResult.explanation() output is paste-safe per its docstring.
Add regression test test_compact_repr_escapes_quotes_and_backslashes.
- Update three tests using project="p" (1 char) to use "proj01" (6
chars) so they reach the dataset/name validation they target.
* bd_1-scaffolding-8xk.13: untrack uv.lock (bark worktree artifact)
uv.lock was inadvertently included in the prior Quality Gate commit;
it's bark's worktree scaffolding output and isn't part of the SignalForge
build (we use Hatchling + pip, not uv). Add to .gitignore.
* bd_1-scaffolding-8xk.13: Quality Gate — fix 2 findings from review pass 3
- Centralise BQ string-literal escaping in _sql_safety.escape_bq_string_literal
(handles backslash, single-quote, newline, CR, tab, NUL); use it from both
_render_partition_filter and _test_result_repr. Newlines/control chars in
partition values previously caused BigQuery syntax errors at execution
time; the new helper renders all of them as escape sequences. Regression
test test_render_partition_filter_escapes_newlines_and_tabs.
- Wrap BigQueryAdapter.__exit__ in try/finally so a flush failure during
clean exit cannot leave the adapter in a half-cleaned state — the next
`with` block must start from empty caches per DEC-025. Regression test
test_context_manager_exit_clears_caches_even_if_flush_raises.
- Document Pass 3 finding 3 (drift detector self-consistency) as a v0.2
follow-up in docs/warehouse-adapter-ops.md alongside the legacy
domain-scoped project ID gap.
* bd_1-scaffolding-8xk.13: Quality Gate — fix phantom API from review pass 4
UnknownTableSizeError.default_remediation referenced an undefined
adapter.refresh_table_metadata method, breaking Architectural
Commitment #5 (explainable diffs — the remediation must actually work).
- Implement BigQueryAdapter.refresh_table_metadata(table) — drops the
cached Table for one ref so the next operation re-fetches num_rows.
No-op outside an active context.
- Add two regression tests: cache invalidation roundtrip + no-op
outside context.
- Drop the now-shipped GCP project-ID grammar entry from the v0.2
follow-ups list (it was completed in QG pass 1).
* bd_1-scaffolding-8xk.14: Patterns & Memory — warehouse-adapters rule + CLAUDE.md surface
Captures the conventions established by issue #3 into .claude/rules/warehouse-adapters.md:
ABC + lazy-import factory, _client.py pyright containment, expect_* fake API,
deterministic hash-mod sampling with fail-loud sizing, identifier validation at
construction time, _default_job_config defaults (use_query_cache=False non-negotiable),
typed error hierarchy with remediation + repr-quoted user input, and the explicit
US-014 decision to keep _path_safety duplicated rather than extracted.
Updates CLAUDE.md Repository status (two issues -> three) and Public API surface to
include signalforge.warehouse re-exports.
* PR #16: Address Copilot review comments
Code:
- TableRef: add `qualified_name` property (dialect-neutral `[project.]dataset.name`).
Used by sampling errors so `.table` is a stable identifier rather than the dataclass repr.
- bigquery.sample_rows: validate `n > 0` to prevent ZeroDivisionError on bucket sizing.
- Map google.api_core exceptions with table identity from the call site:
`map_bq_exception` now reads `context["table"]` so TableNotFoundError.table /
ColumnNotFoundError.table carry the qualified name instead of the truncated message.
Column name is extracted from BigQuery's "Unrecognized name: foo" via regex.
- base.from_profile: replace `or 100_000_000` with explicit `is None` so an explicit
`maximum_bytes_billed: 0` in the dbt profile is honoured.
- profiles.load_profile: thread the resolved profiles.yml path into ProfileNotFoundError;
ProfileTargetNotFoundError now lists the available output names in its remediation
and exposes them via `.available` and `.profiles_path` fields.
Docs:
- warehouse-adapter-ops.md: correct the column_stats batching description (first call
flushes every queued column for the table in a single query); narrow the
BytesBilledExceededError doc to match what the mapping actually populates.
- .claude/rules/warehouse-adapters.md: fix the FakeBigQueryClient snippet to match the
real `expect_*` API; acknowledge the profiles.yml soft-size WARNING.
Tests:
- test_models: TableRef.qualified_name with and without project.
- test_bigquery_unit: n<=0 ValueError; .table stable identifier on UnknownTableSize /
SamplingRequiresPartitionFilter / TableNotFound; .column extraction on ColumnNotFound.
- test_base: from_profile honours explicit maximum_bytes_billed=0.
- test_profiles: ProfileTargetNotFoundError exposes .available, .profiles_path, and
the remediation message contains every available target name.
* PR #16: ruff format the new qualified_name test
* 4: PII safety layer (#18)
* Add super plan for #4: PII safety layer
Phase 1 locks library-only scope (no CLI, no LLM client — those land in
#9 and #5). Phase 2 surfaced the load-bearing finding that schema-only
mode leaks PII via column NAMES; Phase 3 resolves it with stable
blake2b-hash placeholders. 26 decisions captured across config, fail-
closed audit semantics, AuditEvent reproducibility (signalforge_version
+ policy_hash + audit_schema_version), extra=forbid on config-shaped
models, and the SafetyPolicy.with_mode() seam #9 will use.
Detailed Breakdown is 14 stories: scaffolding -> fixtures -> errors ->
models -> SafetyPolicy -> config loader -> audit -> redact -> aggregate
-> request builder -> public API + drift detector + AST scan -> docs ->
quality gate -> patterns. TDD specified for every story with non-trivial
business logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Update phase to published with PR #18 link
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Devolve plan to beads (epic + 14 tasks)
Phase: devolved. Epic bd_1-scaffolding-o6f live; 14 tasks wired into the
architecture-order DAG; bd ready confirms US-001 is the single entry
point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Populate Beads Manifest section; remove stale placeholder duplicates
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-0ix: Scaffold signalforge.safety subpackage + add safety pytest marker
Empty __init__.py placeholder; full public re-exports land in US-011.
New pytest marker enables `pytest -m safety` once tests exist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-o64: Add safety-layer test fixtures (signalforge.yml variants, manifest with PII meta, audit JSONL sample)
Hand-author the fixture corpora consumed by the upcoming PII-safety
stories (US-005..US-011): eight signalforge.yml variants exercising
the locked safety: top-level shape (DEC-025), the redact extend/replace
mutual exclusion (DEC-017), the sampling-mode flag (DEC-021), and the
DEC-013 path-traversal guard; a hand-derived manifest fixture carrying
all four column-level PII opt-out signals plus a model-level signal
(DEC-026); a one-line audit JSONL sample matching the locked AuditEvent
shape (DEC-005 + DEC-014); and a deterministic regeneration script that
will swap to the typed AuditEvent model once US-004 lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-rix: Add signalforge.safety.errors with 10-class hierarchy
SafetyError base + 9 typed subclasses; mirrors WarehouseError/ManifestError
patterns (default_remediation ClassVar, ↳ Remediation rendering, repr-quoted
user input via _format_value helper).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-agc: Add signalforge.safety.models (SamplingMode, RedactionRecord, AuditEvent, LLMRequest)
Frozen Pydantic v2 models with deep-immutable tuple sequences (DEC-022).
AuditEvent carries reproducibility fields (signalforge_version, policy_hash,
audit_schema_version=1) per DEC-014. LLMRequest docstring warns against
direct construction (audit-completeness convention; AST scan in US-011).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-pfj: Add SafetyPolicy + _resolve_redact_patterns + _compute_policy_hash
Frozen Pydantic v2 with extra=forbid; default mode=SCHEMA_ONLY; case-insensitive
mode load; @model_validator resolves redact.extend/replace mutual exclusion;
pattern-injection rejection (empty/*/?); with_mode() factory for #9's CLI;
sample-mode WARNING; deterministic 16-hex policy hash for AuditEvent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-mg2: Add signalforge.safety.audit (fail-closed JSONL writer)
O_APPEND atomic-append with fsync; mkdir parent at 0o700; file at 0o600.
PIPE_BUF size cap (4000 bytes) raises AuditRecordTooLargeError. Any I/O
exception propagates as AuditWriteError (DEC-011 fail-closed). Logger uses
lazy-format with json.dumps to avoid ANSI/log-injection (DEC-022).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-ly6: Add signalforge.safety.config (load_safety_config + path safety)
Full DEC-016 error contract: explicit-path miss raises ConfigNotFoundError;
implicit miss / empty / missing-safety-key falls through to defaults;
malformed YAML / non-mapping / schema-invalid raise typed errors. yaml.safe_load
only. audit_path canonicalised + reject .. segments + require inside project_dir
(DEC-013). _path_safety.py copied from warehouse layer per duplication precedent.
* bd_1-scaffolding-y47: Add signalforge.safety.redact (classify + redact_rows + hash_column_name)
_classify_column returns RedactionRecord | None; precedence column>model;
case-insensitive tag matching; meta.contains_pii truthy coercion (DEBUG-log).
Pattern match case-insensitive (lowercase both sides). hash_column_name uses
blake2b-4 for stable 8-hex-char placeholders (DEC-010). redact_rows replaces
values with '<REDACTED>' constant (no mutation). redact_column_names
substitutes hashed names. Suspicious-unmatched-column WARNING heuristic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-3z5: Add aggregate_columns + FakeAdapter
aggregate_columns wraps adapter.column_stats inside `with adapter:` (DEC-008
batching). Redacted columns return None keyed by hashed name; non-redacted
keyed by real name. FakeAdapter mirrors warehouse FakeBigQueryClient's
expect_* API; never MagicMock. ColumnNotInModelError on unknown column.
Empty columns list short-circuits without ever opening the adapter context.
When every requested column is redacted, the warehouse is never touched.
The PII fixture's `database: "dev"` was bumped to `sf-demo-proj` so it
satisfies BigQuery's project-ID grammar (6-30 chars, lowercase start) — the
classify tests don't depend on the value, but TableRef.from_model now does.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-969: Add build_llm_request + default-mode regression suite
Single entry point per DEC-009: classifies columns, dispatches per-mode
(schema-only zero warehouse calls — DEC-012(c)), writes AuditEvent before
returning (DEC-011 fail-closed). AuditEvent carries signalforge_version,
policy_hash, audit_schema_version=1 (DEC-014). policy_flags populated from
policy state (sample_mode_enabled, redaction_disabled, audit_path_overridden).
Three default-mode regression tests cluster DEC-012 at policy/config/request
layers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-2fb: Wire signalforge.safety public API + drift detector + AST scan
__init__.py re-exports the documented surface (DEC-001). StrictAuditEvent
drift detector validates the committed JSONL fixture and asserts field-set
parity with production AuditEvent (DEC-026). AST scan rejects LLMRequest
construction outside request.py — the audit-completeness convention from
DEC-020(a). Includes negative test that the AST visitor catches a planted
violation.
Note: ``_path_safety`` is asserted absent from ``__all__`` rather than
``dir()``: Python attaches imported submodules to the parent package's
namespace once any sibling (``config.py``) imports them, regardless of
``__init__.py``. Intent (private-helpers-stay-private) preserved via the
``__all__`` check, mirroring the warehouse package's pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-gx3: Add docs/safety-ops.md + README 'Data safety' section + .gitignore
Operational reference for the PII safety layer matching manifest- and
warehouse-ops doc precedent: default posture, modes, signalforge.yml schema,
redaction patterns, four opt-out signals + precedence, column-name redaction
(DEC-010 blake2b), audit JSONL schema with audit_schema_version=1,
sensitivity caveat (column names are plaintext), rotation as user
responsibility, debugging (logger levels), typed-error reference, CLI
integration note pointing at #9.
* bd_1-scaffolding-8av: Quality gate — fix bugs from code review
Three blockers from quality-gate review:
1. LLMRequest.aggregates was dict[str, ColumnStats|None] — frozen=True
only blocks attribute reassignment, not dict mutation. Downstream
consumers (#5) could rewrite values after the audit log was written,
desyncing the audit from what the LLM actually saw. Switched to
tuple[tuple[str, ColumnStats|None], ...] for transitive immutability
(DEC-022). Conversion happens at the request-builder boundary;
aggregate_columns' public dict return is unchanged.
2. SafetyPolicy.with_mode() used model_copy(update=...) which silently
skipped @model_validator(mode="after"). Calling
policy.with_mode(SamplingMode.SAMPLE) — the documented CLI override
seam (#9) — silently enabled sample mode without emitting the
DEC-021 WARNING. Now goes through model_validate so the warning
fires every time, regardless of construction path.
3. SafetyPolicy._normalise_mode returned non-string non-enum values
unchanged, letting Pydantic raise a generic ValidationError instead
of the typed InvalidSamplingModeError. Now every invalid type
raises the typed error so the safety-layer hierarchy stays
homogeneous.
Plus: documented the .signalforge/ pre-existing-permissions caveat in
docs/safety-ops.md (mkdir(exist_ok=True, mode=...) does not tighten
pre-existing directories — user must verify pre-deploy).
Twelve new regression tests:
- test_safety_policy_with_mode_sample_emits_warning (DEC-021)
- test_safety_policy_with_mode_schema_only_emits_no_warning
- test_safety_policy_mode_non_string_non_enum_raises_invalid_sampling_mode_error
(parametrised across 7 bad types: int, float, None, list, dict, tuple, object)
- test_llm_request_aggregates_is_tuple_of_tuples_when_present (DEC-022)
- test_llm_request_aggregates_immutable_when_none
- test_llm_request_aggregates_field_reassignment_blocked_by_frozen
Validation: 423 passed (up from 411), ruff/pyright/format all clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-51d: US-014 — Patterns & Memory (rules/safety-layer.md + CLAUDE.md + bd remember)
New .claude/rules/safety-layer.md distils the load-bearing patterns from
issue #4: fail-closed audit semantics, column-name redaction via stable
blake2b-4 hash, AuditEvent reproducibility fields (signalforge_version +
policy_hash + audit_schema_version), extra=forbid on config-shaped models
vs extra=ignore on read-back, the four opt-out signals + precedence,
ANSI-safe lazy-format logger, AST audit-completeness scan, model_copy
re-validation gotcha, signalforge.yml top-level namespace.
CLAUDE.md updated: 'Repository status' bullet for issue #4, 'Public API
surface (v0.1)' entries for signalforge.safety (SamplingMode, SafetyPolicy,
LLMRequest, load_safety_config, build_llm_request, SafetyError hierarchy).
Four bd remember entries:
- schema-only mode redacts column names too
- audit writes are fail-closed
- Pydantic v2 extra=forbid (config) vs extra=ignore (read-back) split
- model_copy doesn't re-run @model_validator(mode='after')
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address Copilot PR #18 review (3 real bugs + 2 doc fixes)
Five Copilot review comments. Three real bugs and two doc inaccuracies:
1. config.py:101 (real bug) — load_safety_config's default-fallback
branches returned SafetyPolicy() with the relative
Path('.signalforge/audit.jsonl') field default, making the audit log
resolve relative to CWD not project_dir. Now every default-fallback
branch (no file / empty file / comments-only / missing safety: key /
safety: present but no audit_path) canonicalises DEFAULT_AUDIT_PATH
against project_dir before passing to SafetyPolicy. Symmetric with
the user-override path.
2. config.py:136 (real bug) — audit_path: 123 (or [a,b], or true) crashed
inside Path(audit_path_raw) with TypeError, leaking a non-SafetyError.
Now type-checked at the gate: non-(str|os.PathLike) raises
InvalidConfigError with a clear remediation pointing at YAML quoting.
3. config.py validator path (real bug) — Pydantic's extra="forbid" raises
ValidationError with type='extra_forbidden', which the loader was
wrapping as the generic PolicyValidationError. DEC-026 specified
UnknownConfigKeyError for typos like `redacts:` or `mode_:`. Now the
loader walks the Pydantic error list a second time and translates
extra_forbidden into the typed UnknownConfigKeyError so the contract
in safety-ops.md actually holds.
4. models.py:70 (doc bug) — RedactionRecord docstring claimed records are
emitted for every column considered. They aren't — only redacted
columns produce records (None pass-through for non-redacted). Updated
docstring to match.
5. safety-ops.md:72 (doc bug) — aggregate-only example showed
request.aggregates as a dict. After the Quality-Gate fix that made
LLMRequest.aggregates a tuple-of-tuples (DEC-022 transitive
immutability), the example was stale. Updated.
The sixth Copilot comment (request.py:5) flagged stale PR metadata
('Phase: detailing (awaiting approval)' / 'Review the plan in this PR')
that I had already fixed via REST API after the PR was opened. The
title is now '4: PII safety layer' and the description reflects the
shipped implementation. Will mark that comment as outdated.
Eight new regression tests:
- test_load_safety_config_no_file_default_audit_path_is_inside_project
- test_load_safety_config_empty_file_default_audit_path_is_inside_project
- test_load_safety_config_missing_safety_key_default_audit_path_is_inside_project
- test_load_safety_config_no_audit_path_in_yaml_canonicalises_default
- test_load_safety_config_typo_fixture_raises_unknown_config_key_error
(renamed + tightened from generic SafetyError check)
- test_load_safety_config_top_level_typo_raises_unknown_config_key_error
- test_load_safety_config_audit_path_int_raises_invalid_config_error
- test_load_safety_config_audit_path_list_raises_invalid_config_error
- test_load_safety_config_audit_path_bool_raises_invalid_config_error
Validation: 431 passed (up from 423), ruff/pyright/format all clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 5: LLM draft pipeline (#19)
* Add super plan for #5: LLM draft pipeline
Plan doc covers Discovery, Architecture Review, 27 DECs across Refinement,
and an 18-story Detailed Breakdown ready for devolve.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Devolve #5 plan to beads (epic + 18 tasks)
Update plan-doc Phase marker to 'devolved' and fill in the Beads Manifest
section with epic ID, the 18 task IDs (one per US-001..US-018), and the
dependency graph.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-g11: US-001 — Subpackage scaffolding + anthropic dep + pytest markers
Create empty signalforge.llm and signalforge.draft subpackages (placeholder
__init__.py only; __all__ re-exports land in US-013). Add
anthropic>=0.50,<1.0 to [project.dependencies]. Register pytest markers llm,
draft, and anthropic; extend default addopts to also exclude the anthropic
real-API smoke marker so default CI runs llm/draft but skips anthropic.
Traces to: DEC-001, python-build.md, testing-signal.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-wsb: US-002 — Test fixtures (yml + manifest + golden response samples)
* bd_1-scaffolding-4e2: US-003 — signalforge.llm.errors hierarchy
* bd_1-scaffolding-zea: US-004 — signalforge.llm.models LLMResult
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-806: US-005 — signalforge.llm._client SDK shim (DEC-012 confinement)
* bd_1-scaffolding-hhn: US-006 — signalforge.llm.client.call_anthropic centralized seam
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-wwx: US-007 — signalforge.draft.errors with bad-JSON envelope
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-nlz: US-008 — signalforge.draft.models CandidateSchema family + schema_version
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-buv: US-009 — signalforge.draft.config DraftConfig + load_draft_config
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-0kq: US-010 — signalforge.draft.prompts in-code template + version hash + envelope + mode-varying section
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-g27: US-011 — signalforge.draft.parser JSON validation + anchor-contract validator
* bd_1-scaffolding-mtg: US-012 — signalforge.draft.audit LLMResponseEvent + fail-closed writer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-9na: US-013 — signalforge.draft.schema integration + DraftOutcome + public API
* bd_1-scaffolding-n98: US-014 — Audit-completeness AST scans + drift detector + cache-stability snapshot + grep gate
* bd_1-scaffolding-eup: US-015 — Real-API smoke test (@pytest.mark.anthropic)
Add tests/draft/test_smoke_real_api.py and the
tests/fixtures/draft/smoke_manifest.json fixture it consumes. Gated by
the ``anthropic`` marker (registered in pyproject.toml from US-001),
excluded from default CI by the ``-m 'not bigquery and not anthropic'``
filter, runnable locally with ``pytest -m anthropic`` when
``ANTHROPIC_API_KEY`` is set.
Path A (wire test): the cached manifest summary for the synthetic
``simple_orders`` model is ~100 tokens, far below Haiku's
2048-token minimum and Sonnet's 1024-token minimum. The pre-send
``count_tokens`` call hits the real Anthropic API (proving SDK +
auth + transport), then ``call_anthropic`` raises
``LLMCacheTooSmallError`` before any ``messages.create`` is issued.
This is documented in the test docstring as a deliberate v0.1
constraint; when the smoke fixture grows above 2048 tokens the test
can be upgraded to Path C (full round-trip) without changing its
marker or invocation.
Fixture exercises the no-neighbours code path of
``_render_manifest_summary`` (empty ``depends_on.nodes``, empty
``refs``).
Validation: ruff/ruff-format/pyright clean. ``pytest`` default still
reports 608 passed; ``deselected`` count goes from 6 to 7 to reflect
the new ``anthropic``-marked test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-bhu: US-016 — docs/draft-ops.md + README + CLAUDE.md
* bd_1-scaffolding-273: US-017 — Quality gate fixes from code review pass 1
Three real bugs surfaced and fixed (pass 2 truncated by org usage limit;
all four validation checks green).
1. parser._validate_anchor_contract: a CandidateColumn whose `name` was
not in `model_columns` slipped through (the membership check lived
only inside an `elif` after the parent-column-mismatch branch). Now
the column name itself is validated, and the nonexistent-column check
on each test is independent of the parent-column-mismatch check so a
hallucinated column surfaces both violations.
2. client.call_anthropic: the cache-anomaly WARNING fired on every cache
HIT. `cache_creation == 0` and `cache_read > 0` is the normal healthy
case (the cache was created on a prior call). The warning now requires
both creation AND read to be zero — the genuine no-op signal.
3. prompts._render_dynamic_block: a Model.raw_code containing the
literal `</MODEL_SQL>` would terminate the prompt-injection envelope
early and let downstream content escape the data fence. New typed
error PromptEnvelopeBreachError; render is now refused before any
LLM call.
Tests: 608 → 611 (one new test per fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-x3t: US-018 — Patterns & Memory (.claude/rules/llm-drafter.md + bd remember)
Distil load-bearing patterns from issue #5 into a project rule that
future contributors (human + Claude) consult before touching
signalforge.llm or signalforge.draft. Mirrors safety-layer.md shape.
Sections: SDK seam confinement (DEC-012) | retry test-determinism via
module aliases (DEC-004) | fail-closed response audit (DEC-006/008/013)
with the propagation-IS-the-defence rule | <MODEL_SQL> envelope + breach
guard (DEC-007) | cached-block scope + 8000-token cap (DEC-009) |
cache-anomaly dual-zero rule (post-QG fix to DEC-014) | whole-draft
fail-loud anchor contract with collected violations (DEC-003/022) |
ANSI-safe lazy-format logger gate (DEC-011) | four AST audit-completeness
scans (DEC-013) | signalforge.yml top-level llm: namespace (DEC-027).
Plus 5 persistent bd memories (issue-5-llm-*) so cross-conversation
recall is anchored on the load-bearing patterns, not the file paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-273: PR review fixes (Copilot + CodeRabbit, 12 threads)
All 12 review threads addressed; none deferred. Two require behaviour
notes; the rest are mechanical hardening of existing seams + tests.
PROMPT_VERSION rotated 4800590d3d749955 -> 1c55806467984090 because
the system prompt's claim that "tests without rationale are rejected
by the parser" was softened (the parser does not, in fact, enforce it
— DEC-026 documented this as a soft constraint owned by the grader at
#7). Cache-stability test pinned to the new value.
Real bug fixes:
- src/signalforge/draft/schema.py: sent_sql_hash now hashes
Model.raw_code (not the dynamic block which carries envelope + data
section). Drift would have broken correlation with raw_code.
- src/signalforge/draft/models.py: schema_version: Literal[1] = 1 so
payloads with schema_version != 1 fail validation rather than
silently passing.
- src/signalforge/llm/client.py: count_tokens now MAPS Anthropic
exceptions to typed LLMError subclasses (no retry on the probe
call — consuming the create-budget on a probe failure would let one
blip exhaust the budget). Plus per-class retry counters
(attempt_429 / attempt_5xx / attempt_conn) so one failure class
cannot consume another's budget; total_attempts drives the backoff
math + WARNING log so delays remain monotonic across mixed types.
- src/signalforge/draft/prompts.py: removed the false claim that the
parser rejects rationale-less tests. New phrasing matches actual
enforcement (parser is permissive; grader scores).
Hardened test infrastructure:
- tests/llm/test_logger_grep_gate.py + tests/llm/test_client.py: regex
now matches every f-string prefix permutation (f, F, rf, fr, rF,
FR, etc.) AND optional whitespace after the opening paren AND single
or double quotes. Previous regex only caught (f". Bypassable by
switching quote style.
- tests/test_audit_completeness.py: AST scan handles import aliasing
(import anthropic as a; a.Anthropic(...)) AND direct-symbol imports
(from anthropic import Anthropic; Anthropic(...)). Previous matcher
could be silently bypassed.
- tests/test_audit_completeness.py: exclusion lists now path-relative
(request.py / _client.py / audit.py at root of scan dir) rather than
basename-only — prevents accidental shadowing of nested files with
the same basename.
Trivial fixes:
- README.md: MD040 fence language (text).
- tests/fixtures/draft/regenerate.sh: comment now reflects current
reality (US-015 smoke test exists; refresh is still manual until
fixture grows past cache minimums).
New tests:
- test_call_anthropic_per_class_budgets_do_not_cross_consume: pins the
invariant that one class's failures cannot consume another's budget.
612 passed (was 611 + 1 new).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 6: Test prune engine (#20)
* 6: super-plan for prune engine
Phase-4 plan for issue #6 — drop always-pass and known-clean-fail
candidate tests against warehouse data. Covers the load-bearing
"signal over volume" commitment from CLAUDE.md.
Architecture review surfaced two pivots resolved in DEC-012/DEC-013:
the deterministic-sample predicate (TO_JSON_STRING(t)) reads every
column, so US-003 live-verifies the cost model against bigquery-
public-data before locking strategy; the warehouse adapter's
QueryJobConfig.job_timeout_ms plumbing folds in as US-002 (~10 LOC)
so the per-test budget actually enforces.
28 decisions, 16 stories (14 implementation + Quality Gate +
Patterns & Memory). Stories trace to DEC-### and embed TDD on
every logic-shaped surface (compiler, engine, audit, errors,
models, config).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 6: bump plan phase to published
PR #20 is open; phase marker updated for re-invocation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 6: devolve plan — beads manifest
Epic bd_1-scaffolding-y8y + 14 implementation tasks + QG + PM.
Dependencies wired so `bd ready` returns only US-001 at start.
Phase marker bumped to devolved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-y8y.1: scaffold signalforge.prune subpackage
Empty docstring-only stubs for engine, compiler, models, errors,
audit, and config modules. Subsequent stories (US-002 ... US-014)
land the actual logic.
Plan: plans/super/6-prune-engine.md (DEC-001).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-y8y.4: PruneResult / PruneDecision / primitives
Frozen Pydantic v2 models for the prune layer's read-back surface.
DropReason and Scope discriminator literals. Computed-property
aggregates (kept_decisions, dropped_decisions, kept_count, etc.) for
the diff renderer (#8). PruneDecision carries the typed CandidateTest
discriminated union (DEC-004), not a loose dict.
Plan: plans/super/6-prune-engine.md (DEC-003, DEC-004, DEC-014, DEC-015).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-y8y.6: PruneError hierarchy
Six typed exception classes: PruneError, PruneConfigError,
PruneTrustedModelNotFoundError, PruneTimeoutError,
PruneAuditWriteError, PruneAuditRecordTooLargeError. Each carries
default_remediation; __str__ renders message + remediation; user
input is repr()-quoted to defeat ANSI injection.
Introduces signalforge.errors.SignalForgeError as the project-wide
root so PruneError(SignalForgeError) wires per DEC-006. Existing
layer roots (SafetyError, DraftError, WarehouseError, ...) stay
untouched per task scope; future stories can rebase them.
Plan: plans/super/6-prune-engine.md (DEC-006, DEC-022).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bd_1-scaffolding-y8y.6: drop spurious signalforge.errors module
The worker's first commit added signalforge/errors.SignalForgeError
to satisfy the plan's PruneError(SignalForgeError) note, but every
existing layer (Safety/Draft/Warehouse/Manifest) subclasses Exception
directly with no shared root. Adding a project-wide root for prune
alone left the codebase inconsistent.
Match the established pattern: PruneError(Exception). The plan note
referencing SignalForgeError was a planning bug; the prece…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #2 — read
manifest.jsonand resolve a single model.This PR is the full implementation of the issue, not just a plan. The original draft contained the planning doc only; subsequent commits delivered all 9 stories from the plan. The PR description is now updated to reflect the shipped state.
What's in this PR
Plan + research (
plans/,docs/research/):plans/super/2-manifest-loader.md— full /super-plan output (Phases 1–7).docs/research/— seven dbt research files vendored fromclauditor/docs/temp/(DEC-006).Implementation (
src/signalforge/manifest/):errors.py—ManifestErrorhierarchy (7 typed exceptions) withremediation: strrendered in__str__(DEC-013, DEC-014).models.py— Pydantic v2Manifest/Model/Column/Ref/Config/DependsOn. Frozen,extra="ignore",populate_by_name=True. Field validators onunique_idandraw_code(DEC-001, DEC-011, DEC-016, DEC-017).loader.py—load(project_dir, manifest_path=None) -> Manifest. Symlink-hardened path resolver (DEC-007, including the default-path canonicalisation fix from the quality gate). Soft 200 MB warning (DEC-008). Schema-version detection v9–v12 with feature-sniff fallback. Free-function resolvers wrapped as thin methods onManifest(Option B, DEC-017).__init__.py— public__all__:load,Manifest,Model, plus the seven error classes.Test corpus (
tests/):tests/manifest/{test_errors,test_models,test_loader,test_public_api}.py— 67 tests across unit / integration / error markers.tests/fixtures/dbt_project_small/— real tiny dbt project + 4 pre-generated manifests (v9–v12).tests/fixtures/dbt_project_medium/— 55-model synthesised project + v12 manifest.tests/fixtures/error_paths/— 5 hand-crafted error fixtures.tests/fixtures/regenerate.sh+tests/fixtures/README.md— multi-version regen via ephemeraluvx dbt-core==X.Y.*.Docs + conventions:
docs/manifest-loader-ops.md— memory profile, supported versions, error reference (DEC-008).CONTRIBUTING.md— Test markers + Regenerating fixtures subsections.README.md— v0.1 status callout updated..claude/rules/manifest-readers.md(new) — Pydantic v2 + symlink-hardening + remediation conventions for future external-format readers..claude/rules/testing-signal.md— appended fixture-regen + drift-detector patterns.CLAUDE.md— issue Read dbt manifest.json and resolve a single model #2 marked shipped; new "Public API surface (v0.1)" section.Dependencies (
pyproject.toml):pydantic>=2.5,<3(first runtime dep).dbt-core>=1.8,<2(fixture regeneration only — not a runtime dep).unit,integration,error.Quality gate
Per the plan's US-008, ran 4 sequential code-reviewer passes. Pass 2 (adversarial) caught two real bugs in the path-hardening logic — both fixed with regression tests:
target/manifest.jsonbypassed_canonicalise_path(a symlink at the default path could escape the project root).Path.resolve()raisesRuntimeErroron symlink cycles regardless ofstrict=; this leaked instead of becoming a typedModelPathOutsideProjectError.Three new fail-capable regression tests cover both. CodeRabbit was not available in this environment.
Validation