fix: harden rewrite replacement and leak metrics

[binaryaaron]

Summary

• apply replacement-map entries to validated source spans before rewrite generation and carry that baseline through repair
• emit schema-v2 metrics that distinguish map entries, targeted/applied/skipped spans, unique leaked values, and associated source occurrences
• separate post-validation detection artifacts from materialized final entities and keep legacy schema-v1 ingestion isolated
• consolidate local and generated-map DataFrame orchestration around one span-application helper, without iterrows() or per-row Series construction

Why

The schema-v1 leak metric used raw substring matching for longer values and counted every same-valued source entity when one match appeared in output. This produced false positives such as lawyer in lawyers. Adjacent replacement metrics also described map membership rather than actual span execution, and detection sidecars exported post-validation detections under final_* names.

The rewrite path additionally relied on the model to apply deterministic replacement-map entries. This change uses the existing offset-based replacement boundary, fails closed when required mappings cannot be applied, and prevents the repair loop from returning to the original sensitive baseline. It supersedes the global-regex runtime approach proposed in #208.

Both local replacement and LLM-generated replacement maps now use the same DataFrame orchestration helper. The helper iterates aligned columns directly, preserves row order and custom column names, and records replacement text and application telemetry together.

Compatibility

• new measurements emit schema version 2 and explicit metric names
• strict ingress continues to accept schema-v1 records
• mixed schema snapshots and cross-version metric fields are rejected
• benchmark groups include measurement schema version, so v1 and v2 series are not aggregated together
• legacy artifact sidecars are interpreted as detected metrics; final metrics remain unavailable unless actual final entities are present

Validation

• make check
• make test (1,219 passed, one existing W&B deprecation warning)
• uv run pytest tests/engine/test_replace_strategies.py tests/engine/test_replace_runner.py -q (21 passed)
• git diff --check
• Astnav syntax query confirms zero iterrows() calls in strategies.py

[greptile-apps]

Greptile Summary

The PR applies deterministic replacement-map entries to validated source spans before rewrite generation and carries the sanitized baseline into repair. It also introduces schema-v2 replacement and leak metrics and updates benchmark and W&B tooling to distinguish measurement schema versions.

• Consolidates local and generated replacement-map application around an offset-based helper with application telemetry.
• Fails rewrite rows closed when required replacements cannot be materialized.
• Separates map-entry, span-application, and source-leak cardinalities in measurement schema v2.
• Preserves strict schema-v1 ingestion while rejecting mixed or cross-version metric shapes.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/anonymizer/engine/replace/strategies.py	Introduces the shared offset-based replacement primitive and records targeted, applied, and skipped span telemetry.
src/anonymizer/engine/rewrite/rewrite_generation.py	Materializes replacement-map entries before rewrite generation, rebuilds tagged input, and marks incomplete replacement rows unavailable.
src/anonymizer/engine/rewrite/rewrite_workflow.py	Keeps replacement-unavailable rows out of repair and routes them to human review.
src/anonymizer/engine/rewrite/repair.py	Uses the pre-replaced rewrite baseline rather than restoring original sensitive text during repair.
src/anonymizer/measurement/metrics/replacements.py	Separates replacement-map membership from actual span-application measurements.
src/anonymizer/measurement/metrics/rewrite.py	Replaces ambiguous leak counts with standalone-token unique-value and source-occurrence metrics.
tools/measurement/measurement_tools/wandb_ingress.py	Enforces schema-specific metric fields and rejects mixed-version measurement snapshots.
tools/measurement/analyze_benchmark_output.py	Updates benchmark analysis to interpret and group leak measurements by schema version.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Validated source entities] --> B[Filter replacement-map entries by value and label]
  B --> C[Apply entries to validated source spans]
  C --> D{All required spans applied?}
  D -- No --> E[Mark rewritten text unavailable]
  E --> F[Require human review]
  D -- Yes --> G[Build tagged sanitized baseline]
  G --> H[Generate rewrite]
  H --> I[Evaluate rewrite]
  I --> J{Repair required?}
  J -- Yes --> K[Repair from sanitized baseline]
  K --> I
  J -- No --> L[Materialize final rewrite]
  C --> M[Record targeted, applied, and skipped span metrics]
  L --> N[Record standalone original-value leak metrics]

Loading

_{Reviews (2): Last reviewed commit: "fix: validate duplicated W&B schema vers..." | Re-trigger Greptile}