4: PII safety layer

.claude/rules/safety-layer.md

@@ -0,0 +1,89 @@
+# Safety layer (PII redaction + audit log)
+
+Established by issue #4 (PII safety layer). Apply to every module under `signalforge.safety` and to any new code that constructs an `LLMRequest`, redacts data heading to an LLM, or writes an audit record.
+
+The safety layer sits between the warehouse adapter (#3) and the LLM-drafting pipeline (#5). It enforces SignalForge's deployment-blocker safety posture: schema-only by default, sample is explicit opt-in, every LLM call leaves a durable audit receipt.
+
+## Fail-closed audit semantics (DEC-011)
+
+Any exception inside `audit.write` propagates as `AuditWriteError` from `build_llm_request`. The function never returns an `LLMRequest` whose audit record didn't durably hit disk. Concretely:
+
+- `audit.write` opens with `O_APPEND | O_CREAT | 0o600`, writes one JSONL line, calls `os.fsync`, closes.
+- Catches **no** exceptions internally — `OSError`, `PermissionError`, `IOError`, encoding failures all propagate.
+- Size cap (`_AUDIT_RECORD_LIMIT_BYTES = 4000`) checked **before** any file open, so an oversize record leaves no artifact.
+- `build_llm_request` calls `audit.write` AFTER constructing the request but BEFORE returning it. If audit fails, the partial request is dropped.
+
+An unaudited LLM call is, by definition, PII leaving the warehouse without a receipt — exactly the failure mode this layer exists to prevent. Don't add try/except around audit writes "to be defensive"; the propagation IS the defence.
+
+## Column NAMES leak PII too — redact them with stable hashes (DEC-010)
+
+A column named `customer_ssn` or `john_smith_email` leaks PII via the name itself, even when no values are sampled. The layer redacts NAMES in `schema-only` and `aggregate-only` modes, not just values in `sample` mode.
+
+The hash is `f"col_{blake2b(name.encode(), digest_size=4).hexdigest()}"` — 8 hex chars, deterministic per name, computed once and recorded in `RedactionRecord.hashed_name`. The (real → hashed) mapping is persisted to the audit JSONL so a reviewer can map back; the LLM only ever sees the hash.
+
+When adding a new mode or surface, make sure the LLM-bound payload uses hashed names for every column where `RedactionRecord.redacted is True`.
+
+## AuditEvent reproducibility fields (DEC-014)
+
+Every `AuditEvent` carries three fields that look minor but are load-bearing:
+
+- `signalforge_version: str` — read from `signalforge.__version__` at write time. Lets a reviewer know which code produced the record.
+- `policy_hash: str` — 16-hex-char SHA-256 of the resolved `SafetyPolicy.model_dump_json` (sorted keys, canonical form via the `_compute_policy_hash` helper). Lets a reviewer verify all records in a run came from the same policy.
+- `audit_schema_version: int = 1` — frozen at the constant in production code. Bump when the JSONL schema evolves; v0.2 readers gate on this.
+
+The drift-detector test (`tests/safety/test_drift_detector.py`) pairs production `AuditEvent` (`extra="ignore"`) with a one-off `StrictAuditEvent` (`extra="forbid"`) validated against the committed JSONL fixture. Adding a field to production without updating the strict model OR the fixture breaks the test loudly. Don't bypass.
+
+## Config-shaped models use `extra="forbid"`; read-back models use `extra="ignore"` (DEC-015)
+
+The default in `manifest-readers.md` is `extra="ignore"` for forward-compat. The safety layer **deliberately overrides** for config files:
+
+- `SafetyPolicy`, `_SafetyPolicyContent` (the inner `signalforge.yml` block), `_SafetyConfigFile` → `extra="forbid"`. A typo like `redacts:` (vs `redact:`) in a user-authored YAML file MUST fail loud — silent no-op is exactly the failure mode this ticket exists to prevent.
+- `AuditEvent`, `RedactionRecord`, `LLMRequest` → `extra="ignore"`. These are read back from JSONL files / consumed by downstream stages where forward-compat matters.
+
+Pair every `extra="ignore"` reader-shaped model with a one-off `extra="forbid"` drift detector.
+
+## The four opt-out signals + precedence (DEC-003)
+
+`_classify_column` returns `RedactionRecord | None` based on (in precedence order, first match wins, column-level always beats model-level when both fire):
+
+1. Column-level `meta.signalforge.sample == False` → `column_meta_optout`
+2. Column-level `tags: ["pii"]` (case-insensitive) → `tag_pii_column`
+3. Column-level `meta.contains_pii` truthy → `meta_contains_pii_column`
+4. Model-level `meta.signalforge.sample == False` → `model_meta_optout`
+5. Model-level `tags: ["pii"]` → `tag_pii_model`
+6. Model-level `meta.contains_pii` truthy → `meta_contains_pii_model`
+7. Pattern match against `policy.redact_patterns` (case-insensitive `fnmatch`) → `pattern_match`
+
+Every coercion path emits a DEBUG log when it normalises (e.g., `tags: [PII]` → lowercase, `meta.contains_pii: "yes"` → True). The seven reasons are a `Literal[...]` so audit-log consumers can pattern-match exhaustively. Don't add an eighth without updating both production `RedactionReason` and the drift detector.
+
+## ANSI-safe lazy-format logger (DEC-022)
+
+Every `_LOGGER.{info,warning,debug,error}` call in `signalforge.safety.*` uses lazy-format with `json.dumps()` for any user-controlled string:
+
+```python
+_LOGGER.info("audit event: %s", json.dumps({"unique_id": event.model_unique_id, ...}))
+```
+
+**Never** `_LOGGER.info(f"... {model_unique_id} ...")` — a column name or model id containing ANSI escapes (`\x1b[31m...`) would inject into log viewers. JSON encoding handles this; f-string interpolation does not. Quality-gate validation greps for `_LOGGER.\w+\(f"` and rejects any hits in `src/signalforge/safety/`.
+
+## AST audit-completeness scan (DEC-020(a))
+
+`tests/safety/test_public_api.py::test_llm_request_construction_only_in_request_module` scans every `.py` under `src/signalforge/safety/` (excluding `request.py`) for `Call(func=Name(id="LLMRequest"))` and rejects any hits. The convention is "construct `LLMRequest` only via `build_llm_request`" — the docstring on `LLMRequest` says so, and the AST scan enforces it.
+
+If you add a new module that genuinely needs to construct an `LLMRequest` (e.g., a deserialiser for resumption), update the AST-scan exclusion list AND document the audit-write seam. Don't suppress the test.
+
+## Pydantic v2 `with_mode` re-runs validators (DEC-018)
+
+`SafetyPolicy.with_mode(mode)` does NOT use `model_copy(update=...)` — that path silently skips `@model_validator(mode="after")`, which means going through the documented CLI override seam would silently enable sample mode without emitting the DEC-021 WARNING.
+
+Use `model_validate({**self.model_dump(), "mode": mode})` so every validator re-runs. This was caught by Quality-Gate review and is a regression worth defending against in any future "factory that produces a mutated copy" helpers — `model_copy` is the wrong tool any time a validator side-effect is part of the contract.
+
+## `signalforge.yml` top-level namespace: `safety:` (DEC-025)
+
+The config file's top level is `{ safety: { ... } }`. Other top-level keys (`llm:`, `prune:`, `grade:`, ...) are reserved for future stages and silently ignored by the safety loader. Each stage validates its own subtree independently.
+
+When introducing a new pipeline stage with config, claim its own top-level key. Don't pile config under `safety:` — that violates the namespacing reservation and forces a v2-config migration when you eventually split.
+
+## Reference
+
+`plans/super/4-pii-safety.md` — DEC-001 … DEC-026. `src/signalforge/safety/` — current implementation. `tests/safety/_fake_adapter.py` — `FakeAdapter` + `expect_*` API. `docs/safety-ops.md` — operational reference. `tests/fixtures/safety/manifest_with_pii_meta.json` — fixture exercising all four opt-out signals.

.gitignore

@@ -25,6 +25,9 @@ dist/
 coverage.xml
 htmlcov/
 
+# SignalForge runtime state (audit log lives here; sensitive — see docs/safety-ops.md)
+.signalforge/
+
 # Local-only symlinked / synced skills — ignore all except maintainer-owned skills
 .claude/skills/*
 !.claude/skills/release-manager

CLAUDE.md

@@ -4,20 +4,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Repository status
 
-Pre-alpha. Three issues shipped:
+Pre-alpha. Four issues shipped:
 
 - **#1 (project scaffolding)** — `pyproject.toml` (Hatchling + src layout), `src/signalforge/__init__.py` with `__version__`, smoke test, ruff + pyright + pytest configs, GitHub Actions CI on PRs into `dev` and pushes to `main`, and `CONTRIBUTING.md`.
 - **#2 (manifest loader)** — `signalforge.manifest` subpackage: typed `Manifest` / `Model` (Pydantic v2), `load(project_dir, manifest_path=None) -> Manifest`, single-model resolver by `unique_id` or file path, schema-version tolerance v9–v12, symlink-hardened path canonicalisation, soft 200 MB warning. See `docs/manifest-loader-ops.md` for the operational reference.
 - **#3 (BigQuery warehouse adapter)** — `signalforge.warehouse` subpackage: `WarehouseAdapter` ABC + `from_profile` factory, `BigQueryAdapter` (the only v0.1 concrete adapter), `load_profile` for dbt `profiles.yml`, deterministic hash-mod sampling with fail-loud sizing checks, identifier-validation at construction time, `QueryJobConfig` defaults that pin `use_query_cache=False`. See `docs/warehouse-adapter-ops.md` for the operational reference.
+- **#4 (PII safety layer)** — `signalforge.safety` subpackage: schema-only-default sampling-mode policy, fail-closed audit JSONL writer (`O_APPEND` + `fsync` + size cap), column-name redaction via stable blake2b-4 hashes (DEC-010), four opt-out signals (column meta, model meta, `tags:[pii]`, `meta.contains_pii`) with documented precedence, AST audit-completeness scan rejecting direct `LLMRequest` construction, drift-detector test for `AuditEvent`. `signalforge.yml` config namespace `{ safety: { ... } }` reserved for the safety layer. See `docs/safety-ops.md` for the operational reference and `.claude/rules/safety-layer.md` for the rules distilled from this ticket.
 
-Design is happening in the open on the `dev` branch; remaining feature work (LLM client, prune logic) lands next.
+Design is happening in the open on the `dev` branch; remaining feature work (LLM client #5, prune logic #6, grader #7, diff renderer #8, CLI #9) lands next.
 
 ## Public API surface (v0.1)
 
 - `signalforge.manifest.load`, `Manifest`, `Model`, and the `ManifestError` hierarchy. Documented in `docs/manifest-loader-ops.md`.
 - `signalforge.warehouse.load_profile`, `DbtProfileTarget`, the `WarehouseAdapter` ABC + `from_profile` factory, the `BigQueryAdapter` concrete, the typed value objects (`Dialect`, `BIGQUERY_DIALECT`, `TableRef`, `PartitionFilter`, `ColumnStats`, `TestResult`), and the `WarehouseError` hierarchy. Documented in `docs/warehouse-adapter-ops.md`.
+- `signalforge.safety.load_safety_config`, `SafetyPolicy`, `build_llm_request`, `aggregate_columns`, `redact_rows`, the typed shapes (`SamplingMode`, `RedactionReason`, `RedactionRecord`, `AuditEvent`, `LLMRequest`), and the `SafetyError` hierarchy (10 classes). Documented in `docs/safety-ops.md`.
 
-Internals (`_loader_helpers`, `_sql_safety`, `_path_safety`, `_test_result_repr`, `adapters/_client`, etc.) are `_`-prefixed and not part of the public contract.
+Internals (`_loader_helpers`, `_sql_safety`, `_path_safety`, `_test_result_repr`, `adapters/_client`, `_classify_column`, `_compute_policy_hash`, `_resolve_redact_patterns`, etc.) are `_`-prefixed and not part of the public contract.
 
 ## Validation
 

README.md

@@ -59,6 +59,31 @@ via `WarehouseAdapter.from_profile(profile)`. See
 setup, cost defaults, sampling strategy (and the TABLESAMPLE
 cost-asterisk), `PartitionFilter` use, and the typed-error reference.
 
+## Data safety
+
+Schema-only is the default. The LLM never sees row data unless you
+explicitly opt in via `safety.mode: sample` in `signalforge.yml` (or
+the post-#9 `--mode` CLI flag). Even column *names* that match the
+built-in PII patterns (`*email`, `*phone`, `*ssn`) — or that you flag
+via dbt `tags: ["pii"]` / `meta.contains_pii: true` /
+`meta.signalforge.sample: false` — are replaced with stable hashed
+placeholders (`col_<8 hex>`) before reaching the LLM.
+
+Every LLM call produces one structured record at
+`.signalforge/audit.jsonl` (default; configurable via
+`safety.audit_path`). The file contains plaintext column-name metadata
+and should be treated as sensitive: this repo's `.gitignore` already
+covers `.signalforge/`; the writer creates the directory at `0o700`
+and the audit file at `0o600`. The audit writer is fail-closed — if
+the write fails, the LLM call is aborted (no silent drafts without an
+audit trail). See [docs/safety-ops.md](docs/safety-ops.md) for the
+JSONL schema.
+
+Full reference — mode semantics, the four opt-out signals and their
+precedence, the `signalforge.yml` schema, the audit schema, debugging,
+and the typed-error reference — is in
+[docs/safety-ops.md](docs/safety-ops.md).
+
 ## Roadmap
 
 | Version | Scope                                                                              |