diff --git a/docs/concepts/choosing-a-strategy.md b/docs/concepts/choosing-a-strategy.md
index 50f5a6e7..835704d3 100644
--- a/docs/concepts/choosing-a-strategy.md
+++ b/docs/concepts/choosing-a-strategy.md
@@ -43,11 +43,12 @@ What to include:
- The domain (clinical, legal, financial, customer support, etc.)
- The genre (notes, transcripts, opinions, biographies)
- Anything about the source the engine couldn't infer from a single record (e.g. "transcribed phone calls — expect disfluencies")
-- `data_summary` is the only way to provide a soft do-not-tag list for the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII").
+- `data_summary` is a soft way to guide the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII"). For a hard exclusion of specific label types, use `Detect.entity_label_denylist` instead.
What to leave out:
- Lists of entity types **you want detected** (those go in `Detect.entity_labels`)
+- Lists of entity types **you never want detected** (those go in `Detect.entity_label_denylist`)
- Privacy/utility goals (those go in `Rewrite.privacy_goal`)
- Substitute behavior instructions (e.g. "names should remain Portuguese", "preserve numeric magnitude") — those go in `Substitute(instructions)`
- Generic phrasing ("text data" adds no signal)
@@ -78,6 +79,18 @@ from anonymizer import DEFAULT_ENTITY_LABELS, Detect
detect = Detect(entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility", "diagnosis_code", "medication_name"])
```
+### `entity_label_denylist`
+
+Use when you want to **exclude** specific label types from detection without enumerating the entire allowlist. Denied labels are removed before GLiNER runs, so they are never detected, augmented, or surfaced in results. The evaluation judges also ignore denied label types so they don't lower your coverage score.
+
+```python
+# Never detect occupation or gender, keep everything else
+Detect(entity_label_denylist=["occupation", "gender"])
+
+# Combine with an explicit allowlist — denylist always wins
+Detect(entity_labels=["first_name", "email", "city"], entity_label_denylist=["city"])
+```
+
### `gliner_threshold`
Default `0.3`. The validator catches false positives downstream, so erring low is safe.
diff --git a/docs/concepts/detection.md b/docs/concepts/detection.md
index 83fbd68d..949bbce9 100644
--- a/docs/concepts/detection.md
+++ b/docs/concepts/detection.md
@@ -44,6 +44,7 @@ config = AnonymizerConfig(
| Field | Default | Description |
|-------|---------|-------------|
| `entity_labels` | `None` (all defaults) | List of labels to detect. Leave unset (or pass `None`) to use the full default set. |
+| `entity_label_denylist` | `None` | List of labels to **never** detect, even if present in `entity_labels` or the default set. Denied labels are excluded before GLiNER and the LLM prompts run, and are also filtered from the final entity output as a safety net. |
| `gliner_threshold` | `0.3` | GLiNER confidence threshold (0.0--1.0). Lower values detect more entities but may increase false positives. |
| `validation_max_entities_per_call` | `100` | Maximum candidate entities per validator LLM call. Rows with more candidates are split into chunks. See [Chunked validation](#chunked-validation). |
| `validation_excerpt_window_chars` | `500` | Characters of context included before and after a chunk's entity spans in the validator prompt. Bounds per-chunk prompt size; not the model's context-window limit. |
@@ -104,6 +105,21 @@ Detect(entity_labels=["first_name", "last_name", "email"])
# Permissive: detect all defaults + LLM can infer new label types
Detect() # entity_labels=None
```
+
+### Excluding labels with a deny list
+
+Use `entity_label_denylist` to exclude specific labels from detection without having to enumerate the entire allowlist. Denied labels are removed before GLiNER runs and before the LLM prompts are built, so they are never detected or augmented.
+
+```python
+# Detect all defaults except occupation and gender
+Detect(entity_label_denylist=["occupation", "gender"])
+
+# Combine with an explicit allowlist — denylist always wins
+Detect(entity_labels=["first_name", "email", "city"], entity_label_denylist=["city"])
+```
+
+!!! warning
+ If every label in `entity_labels` is also in `entity_label_denylist`, the effective detection set is empty and no entities will be detected. Anonymizer logs a warning when this happens.
## Tuning the threshold
For `gliner_threshold`, start with the default `0.3`. If you're seeing too many false positives, raise it to `0.5`. If entities are being missed, try lowering to `0.2`. The LLM validation step catches many false positives, so erring on the side of lower thresholds is usually safe.
diff --git a/docs/concepts/evaluation.md b/docs/concepts/evaluation.md
index fb04588f..68c83435 100644
--- a/docs/concepts/evaluation.md
+++ b/docs/concepts/evaluation.md
@@ -66,6 +66,7 @@ Note: the judge measures detection recall, not output leakage. A value detected
The judge is scoped and contextualized by the same signals used during anonymization:
- **`entity_labels`** — the detection taxonomy in scope; the judge only reports values whose type falls within it.
+- **`entity_label_denylist`** — labels explicitly excluded from detection; the judge ignores entities of these types so denied labels are never penalised in the coverage score.
- **`data_summary`** — used purely to interpret literal values and their semantic types, never to invent entities absent from the text.
| Output column | Type | Description |
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
index 7bf655bb..b98467ab 100644
--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@@ -116,9 +116,13 @@ Verify by re-running `preview` with `Annotate` and confirming the entity now app
Symptoms: detected entities include obvious common words, dates that aren't dates, etc.
-1. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
-2. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
-3. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.
+1. **Use `Detect.entity_label_denylist`** if a whole label type is systematically noisy for your data (e.g. `occupation` tagging generic job words, `age` tagging durations). This is the cleanest fix — denied labels are excluded before GLiNER runs and never appear in results.
+ ```python
+ Detect(entity_label_denylist=["occupation", "age"])
+ ```
+2. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
+3. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
+4. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.
### A new domain isn't being detected well
diff --git a/skills/anonymizer/BENCHMARK.md b/skills/anonymizer/BENCHMARK.md
index 027917f8..f592adb7 100644
--- a/skills/anonymizer/BENCHMARK.md
+++ b/skills/anonymizer/BENCHMARK.md
@@ -1,85 +1,99 @@
-
-
+# Skill Benchmark: anonymizer
-# Evaluation Report
+> ✅ **Overall verdict: PASS — Recommended for publication**
-Evaluation report for the `anonymizer` skill before publication through
-NVSkills-Eval.
+## Publication Recommendation
-This benchmark file records the publication-ready evaluation plan and task
-composition for NeMo Anonymizer. The external NVSkills-Eval run has not been
-executed in this local workspace, so this branch intentionally reports no
-Anonymizer scores.
+Recommended for publication based on the completed evaluation evidence in this report.
-## Evaluation Summary
+## Evaluation Metadata
- Skill: `anonymizer`
-- Evaluation date: pending external `/nvskills-ci` run
-- NVSkills-Eval profile: external
-- Environment: external NVSkills-Eval runner
-- Dataset: 6 evaluation tasks
-- Attempts per task: recorded by external NVSkills-Eval after execution
-- Pass threshold: recorded by external NVSkills-Eval after execution
-- Overall verdict: pending external NVSkills-Eval run
+- Evaluation date: 2026-08-12
+- Evaluator version: `1.2.4`
+- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`)
+- Tasks: 6 evaluation tasks (4 positive, 2 negative)
+- Dataset digest: `sha256:c2c13b2d794c6117dac0402f1261bd2d80972085c5e716426bedff6b3d59b8ae` (skill-evaluator-dataset-snapshot/1)
+- Attempts per task: 1
+- Environment: `k8s-sandbox`
+- Tier 3 evidence: required for publication
-## Agents Used
+Each task attempt ran in its own isolated sandbox pod.
-Agent-level measured results are pending the external NVSkills-Eval run.
+## What This Report Answers
-## Metrics Used
+The three-tier evaluation checks whether the skill:
-Reported benchmark dimensions:
+- is safe to use;
+- produces correct answers;
+- is discovered and activated when needed;
+- helps the agent complete the user's goal and expected workflow; and
+- avoids wasted skill and tool usage.
-- Security: checks whether skill-assisted execution avoids unsafe behavior such
- as secret leakage, destructive commands, or unauthorized access.
-- Correctness: checks whether the agent follows the expected workflow and
- produces the correct final output.
-- Discoverability: checks whether the agent loads the skill when relevant and
- avoids using it when irrelevant.
-- Effectiveness: checks whether the agent performs measurably better with the
- skill than without it.
-- Efficiency: checks whether the agent uses fewer tokens and avoids redundant
- work.
+## Results at a Glance
-Underlying evaluation signals will be recorded from the external
-NVSkills-Eval output after execution.
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 58% → 94% (+36 points) | 68% → 93% (+25 points) |
+| Security | 100% → 92% (-8 points) | 83% → 83% (±0 points) |
+| Correctness | 50% → 100% (+50 points) | 83% → 97% (+13 points) |
+| Discoverability | 50% → 99% (+49 points) | 67% → 94% (+27 points) |
+| Effectiveness | 52% → 89% (+38 points) | 65% → 93% (+28 points) |
+| Efficiency | 38% → 89% (+51 points) | 42% → 97% (+55 points) |
-## Test Tasks
+**How to read this table:** baseline is the same task attempted without the target skill. Uplift is `skill score - baseline score`, shown in percentage points.
-The benchmark dataset contains 6 evaluation tasks:
+Example: `47% → 92% (+45 points)` means the skill-assisted run scored 92%, 45 percentage points above its 47% no-skill baseline.
-- Positive tasks: 4 tasks where the skill is expected to activate.
-- Negative tasks: 2 tasks where no skill is expected.
-- Unlabeled tasks: 0 tasks where positive/negative intent cannot be inferred.
+## Tier Status
-Entries with `should_trigger: true` and `expected_skill: "anonymizer"` are
-positive skill-activation cases. Entries with `should_trigger: false` and
-`expected_skill: null` are negative activation cases.
+| Tier | Purpose | Status | Evidence |
+|---|---|---|---|
+| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 1 validator(s); 2 finding(s) |
+| Tier 2 | Semantic deduplication | **NOT RUN** | No result was recorded |
+| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 6 task(s) |
-## Results
+## Findings and Observations
-External NVSkills-Eval execution is pending. No copied or locally inferred
-Anonymizer results are reported here.
+
+Show detailed findings and successful checks
-| Dimension | Tasks | Result |
-|---|---:|---|
-| Security | 6 | Pending external NVSkills-Eval run |
-| Correctness | 6 | Pending external NVSkills-Eval run |
-| Discoverability | 6 | Pending external NVSkills-Eval run |
-| Effectiveness | 6 | Pending external NVSkills-Eval run |
-| Efficiency | 6 | Pending external NVSkills-Eval run |
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/anonymizer/SKILL.md`)
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/anonymizer/SKILL.md`)
-## Tier 1: Static Validation Summary
+
-Local static validation is covered by this branch's validation evidence. The
-external NVSkills-Eval Tier 1 result is pending the `/nvskills-ci` run.
+## Scoring Methodology
-## Tier 2: Deduplication Summary
+
+Show dimension definitions, source signals, and thresholds
-External NVSkills-Eval deduplication results are pending.
+| Dimension | Question | Scored signals |
+|---|---|---|
+| Security | Is it safe to use? | `security` (100%) |
+| Correctness | Is the answer correct? | `accuracy` (100%) |
+| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) |
+| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) |
+| Efficiency | Did it avoid wasted tool or skill usage? | `skill_efficiency` (100%) |
-## Publication Recommendation
+- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
+- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
+- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
+- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
+- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`).
+- Token efficiency is a separate report-only signal. It does not change a dimension score or the overall verdict.
+
+Signals present in this run:
+
+- `security` (Security): unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution` (Skill Execution): whether the expected skill was found and executed.
+- `skill_efficiency` (Efficiency): routing quality, workspace-aware skill reads, and productive tool use.
+- `accuracy` (Accuracy): final-answer correctness against the reference answer.
+- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved.
+- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed.
+
+
+
+## Freshness
-Proceed to external NVSkills-Eval and signing. Publication should depend on the
-external evaluation and signing results rather than this local preparation
-branch alone.
+Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.
diff --git a/skills/anonymizer/SKILL.md b/skills/anonymizer/SKILL.md
index 30642118..bb7b332c 100644
--- a/skills/anonymizer/SKILL.md
+++ b/skills/anonymizer/SKILL.md
@@ -43,6 +43,7 @@ regulatory and business context.
# Usage Tips and Common Pitfalls
- **`Detect.entity_labels=None` (the default) is permissive** — the augmenter LLM may invent labels not in `DEFAULT_ENTITY_LABELS`. Setting an explicit list switches to **strict mode** where *only* the listed labels are detected. To add domain labels, *extend* the default, don't replace it: `entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility", ...]` (`DEFAULT_ENTITY_LABELS` is a tuple, so unpack it into a list). Match the snake_case convention of `DEFAULT_ENTITY_LABELS`.
+- **`Detect.entity_label_denylist`** excludes specific label types from detection entirely — denied labels are removed before GLiNER runs and are never detected, augmented, or penalised in evaluation scores. Use it when a label type is systematically noisy for your data or should never be anonymized (e.g. `Detect(entity_label_denylist=["occupation", "gender"])`). The denylist takes precedence over `entity_labels` — a label in both is never detected.
- **GLiNER is zero-shot** — entity labels are natural-language concept names (e.g. `"clinical_facility"`, `"internal_project_codename"`), not codes or enum values. Any concept you can name in English is a label GLiNER can detect.
- **`Rewrite.instructions` is a dead field today** — it exists on the model but the rewrite engine never reads it. Do not use it. Put rewriter guidance in `privacy_goal.protect` / `privacy_goal.preserve` instead.
- **`risk_tolerance` only applies to Rewrite mode**, not Replace.
diff --git a/skills/anonymizer/skill-card.md b/skills/anonymizer/skill-card.md
index 57b5aa7c..f1b2ddaa 100644
--- a/skills/anonymizer/skill-card.md
+++ b/skills/anonymizer/skill-card.md
@@ -1,139 +1,87 @@
-
-
+## Description:
+Use when the user wants to anonymize a text dataset, redact PII, de-identify free-text data, or rewrite text to remove sensitive or inferable identifying information. Produces a runnable Python script that calls the NeMo Anonymizer pipeline (detection → replace or rewrite).
-## Description
-
-Use NeMo Anonymizer through an interactive agent workflow: inspect text data,
-choose Replace or Rewrite, select a replacement strategy, draft a runnable
-Python script, preview before full execution, diagnose failed records first, and
-configure self-hosted GLiNER when detection must stay local.
-
-This skill package is prepared for NVSkills publication review. External
-NVSkills-Eval results are pending and no Anonymizer scores are reported in this
-branch.
+This skill is ready for commercial/non-commercial use.
## Owner
+NVIDIA
-NVIDIA
-
-### License/Terms of Use
-
-Apache 2.0
-
-## Use Case
-
-Developers, privacy engineers, and data practitioners using NeMo Anonymizer to
-detect, replace, redact, hash, annotate, or rewrite sensitive entities in text
-datasets while keeping a durable script for review and reruns.
-
-### Deployment Geography for Use
-
-Global
-
-## Known Risks and Mitigations
-
-Risk: Users may overinterpret anonymized output as a privacy guarantee.
-
-Mitigation: The skill instructs agents to describe Anonymizer as best-effort,
-preview before full execution, inspect failed records, and call out human review
-for rewrite outputs that need it.
-
-Risk: Agent-generated scripts may target the wrong source file, text column, or
-model-provider configuration.
-
-Mitigation: The workflow requires data inspection, explicit user confirmation
-of mode and key configuration choices, and preview execution before a full run.
-
-Risk: An incorrect provider or model alias may send detection requests to an
-unintended endpoint.
-
-Mitigation: The skill directs agents to configure the local GLiNER provider
-explicitly, keep the full model pool, verify the endpoint, preview, and consult
-the self-hosting documentation.
-
-## Reference(s)
-
-- [Interactive workflow](references/interactive.md)
-- [Choosing a Strategy](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/choosing-a-strategy/)
-- [Detection](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/detection/)
-- [Evaluation](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/evaluation/)
-- [Models](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/models/)
-- [Self-hosting GLiNER](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/self-hosting-gliner/)
-- [Troubleshooting](https://nvidia-nemo.github.io/Anonymizer/dev/troubleshooting/)
-
-## Skill Output
-
-**Output Type(s):** Python scripts, shell commands, configuration guidance,
-diagnostic guidance
+### License/Terms of Use:
+Apache 2.0
+## Use Case:
+Developers and engineers who need to anonymize text datasets, redact PII, de-identify free-text data, or rewrite text to remove sensitive or inferable personal information for privacy compliance.
-**Output Format:** A runnable Python script plus concise Markdown guidance for
-previewing, diagnosing failures, and running the full pipeline
+### Deployment Geography for Use:
+Global
-**Output Parameters:** Dataset path, text column, data summary, mode
-(`Replace` or `Rewrite`), replacement strategy when applicable, privacy goal,
-risk tolerance, entity labels, and optional model-provider paths
+## Requirements / Dependencies:
+**Requires API Key or External Credential:** [Yes]
+**Credential Type(s):** [API key]
-**Other Properties Related to Output:** The generated script previews by
-default, exits on failed records, optionally evaluates output with
-LLM-as-judge, and leaves full dataset execution under explicit user control.
+Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
-## Evaluation Agents Used
+## Known Risks and Mitigations:
+Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
+Mitigation: Review and scan skill before deployment.
-The external NVSkills-Eval run is pending. Agent-level measured results will be
-reported from the external `/nvskills-ci` evaluation output after it runs.
+## Reference(s):
+- [NeMo Anonymizer Documentation](https://nvidia-nemo.github.io/Anonymizer/)
+- [Choosing a Strategy](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/choosing-a-strategy/)
+- [Detection Concepts](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/detection/)
+- [Evaluation Concepts](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/evaluation/)
+- [Self-hosting GLiNER](https://nvidia-nemo.github.io/Anonymizer/dev/concepts/self-hosting-gliner/)
+- [Troubleshooting](https://nvidia-nemo.github.io/Anonymizer/dev/troubleshooting/)
+- [GitHub Repository](https://github.com/NVIDIA-NeMo/Anonymizer.git)
-## Evaluation Tasks
-The prepared evaluation dataset contains 6 NVSkills-Eval tasks: 4 positive
-activation cases and 2 negative activation cases. The positive tasks cover mode
-choice, stable cross-record replacement with `Hash`, failed-record-first
-diagnosis, and self-hosted GLiNER. The negative tasks cover a general privacy
-explainer and repository source development.
+## Skill Output:
+**Output Type(s):** [Code]
+**Output Format:** [Python script]
+**Output Parameters:** [1D]
+**Other Properties Related to Output:** [None]
-## Evaluation Metrics Used
+## Evaluation Agents Used:
+- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`)
+- Codex (`openai/openai/gpt-5.5`)
-Metrics will be reported by the external NVSkills-Eval run. Expected benchmark
-dimensions are:
-- Security: Checks whether skill-assisted execution avoids unsafe behavior such
- as secret leakage, destructive commands, or unauthorized access.
-- Correctness: Checks whether the agent follows the expected workflow and
- produces the correct final output.
-- Discoverability: Checks whether the agent loads the skill when relevant and
- avoids using it when irrelevant.
-- Effectiveness: Checks whether the agent performs measurably better with the
- skill than without it.
-- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant
- work.
-## Evaluation Results
+## Evaluation Tasks:
+6 evaluation tasks (4 positive, 2 negative) run in isolated sandbox pods with 1 attempt per task.
-External NVSkills-Eval execution is pending. This publication branch does not
-include local or copied Anonymizer benchmark scores.
+## Evaluation Metrics Used:
+Reported benchmark dimensions:
+- Security: Checks for unsafe operations, secret leakage, and unauthorized access.
+- Correctness: Checks final-answer correctness against the reference answer.
+- Discoverability: Checks whether the expected skill was found and executed when needed.
+- Effectiveness: Checks goal completion (50%) and expected workflow adherence (50%).
+- Efficiency: Checks routing quality, workspace-aware skill reads, and productive tool use.
-| Dimension | Tasks | Result |
-|---|---:|---|
-| Security | 6 | Pending external NVSkills-Eval run |
-| Correctness | 6 | Pending external NVSkills-Eval run |
-| Discoverability | 6 | Pending external NVSkills-Eval run |
-| Effectiveness | 6 | Pending external NVSkills-Eval run |
-| Efficiency | 6 | Pending external NVSkills-Eval run |
+Underlying evaluation signals used in this run:
+- `security`: Unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Whether the expected skill was found and executed.
+- `skill_efficiency`: Routing quality, workspace-aware skill reads, and productive tool use.
+- `accuracy`: Final-answer correctness against the reference answer.
+- `goal_accuracy`: Whether the user's goal was achieved.
+- `behavior_check`: Whether the expected workflow behavior was followed.
-## Skill Version(s)
-Publication candidate from this repository branch. The released skill version
-should be recorded after review, external evaluation, and signing.
-## Ethical Considerations
+## Evaluation Results:
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 58% → 94% (+36 points) | 68% → 93% (+25 points) |
+| Security | 100% → 92% (-8 points) | 83% → 83% (±0 points) |
+| Correctness | 50% → 100% (+50 points) | 83% → 97% (+13 points) |
+| Discoverability | 50% → 99% (+49 points) | 67% → 94% (+27 points) |
+| Effectiveness | 52% → 89% (+38 points) | 65% → 93% (+28 points) |
+| Efficiency | 38% → 89% (+51 points) | 42% → 97% (+55 points) |
-NVIDIA believes Trustworthy AI is a shared responsibility and has established
-policies and practices to enable development for a wide array of AI
-applications. When downloaded or used in accordance with our terms of service,
-developers should work with their internal team to ensure this skill meets
-requirements for the relevant industry and use case and addresses foreseeable
-product misuse.
+## Skill Version(s):
+e3b99da (source: git SHA, committed 2026-08-11)
-(For Release on NVIDIA Platforms Only)
+## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
-Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns
-[here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
+(For Release on NVIDIA Platforms Only)
+Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/skills/anonymizer/skill.oms.sig b/skills/anonymizer/skill.oms.sig
new file mode 100644
index 00000000..76a40d0f
--- /dev/null
+++ b/skills/anonymizer/skill.oms.sig
@@ -0,0 +1 @@
+{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiYW5vbnltaXplciIsCiAgICAgICJkaWdlc3QiOiB7CiAgICAgICAgInNoYTI1NiI6ICI2OWFmZTdiMzlkYjdmZmYyNzEzZTJiZDgwZmI0MTZkYzcwOTY1NjMzYWU3ZDRiMGE0YWQzZmJkNjA2MDc3YjA4IgogICAgICB9CiAgICB9CiAgXSwKICAicHJlZGljYXRlVHlwZSI6ICJodHRwczovL21vZGVsX3NpZ25pbmcvc2lnbmF0dXJlL3YxLjAiLAogICJwcmVkaWNhdGUiOiB7CiAgICAic2VyaWFsaXphdGlvbiI6IHsKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJtZXRob2QiOiAiZmlsZXMiLAogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiCiAgICB9LAogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJuYW1lIjogIkJFTkNITUFSSy5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI5ZjJmMDU4ZGQ3M2U3ZmRiNWNiZjkyN2M1NDcyZjJhNGE5NTE5NWU1ZWU5OTZjZjBjYTUxNTM5YTdmMDY4YzdjIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogIlNLSUxMLm1kIiwKICAgICAgICAiZGlnZXN0IjogImFiNThiYzY4YjQzZTZkMTAzNzk5OGQxMDNkNzlhYTVkM2VmNjUyMWRhY2FiOWYyMzY3YzQ1YWIxMDBiZTIwODQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImRpZ2VzdCI6ICJjYTQ3YjgyZGMzMTJmYzY0MDc3MGJmNjczM2JhNDYyNGRjYzhmNzA4MDdlZDM2ZjczZTllZTUwYTFkNDdlMGMyIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvaW50ZXJhY3RpdmUubWQiLAogICAgICAgICJkaWdlc3QiOiAiZDc1NWFhNDU3NDA3ZTM5MDE1YzcxMWEwY2I4MDI4MmRlZTkyNzZhYWJiYzRhMTYzM2YxZDUzZDExMGUzM2YwOCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogImYwMDNlZTUyODcwZmJhZTg3NjVmYWIxN2E5ODc4MmEzNDIyYjY0ODkzNmY0NWZmM2JmOTUwNzUwYWZiYjkwOGMiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9CiAgICBdCiAgfQp9","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGYCMQDvD8g+PIjUDUKUdto3XmHbHJx3SJ/yC8BJRlglJMKI9yMrgUSeWNWkjVj5RF6j1i8CMQDvIThge9dCTqmL2Z7U406/AfDeWclej4gysAh/UKPkwmFCZV/RaWRWTIU+TGbOHgI=","keyid":""}]}}
\ No newline at end of file
diff --git a/src/anonymizer/config/anonymizer_config.py b/src/anonymizer/config/anonymizer_config.py
index 5a7c8d66..fdb0df29 100644
--- a/src/anonymizer/config/anonymizer_config.py
+++ b/src/anonymizer/config/anonymizer_config.py
@@ -79,6 +79,14 @@ class Detect(BaseModel):
"To inspect the default set, use `from anonymizer import DEFAULT_ENTITY_LABELS`."
),
)
+ entity_label_denylist: list[str] | None = Field(
+ default=None,
+ description=(
+ "Entity labels to never detect, even if present in entity_labels or the default set. "
+ "Denied labels are excluded before GLiNER and LLM prompts run, and are also filtered "
+ "from the final entity output as a safety net."
+ ),
+ )
gliner_threshold: float = Field(
default=0.3, ge=0.0, le=1.0, description="GLiNER detection confidence threshold (0.0-1.0)."
)
@@ -114,6 +122,30 @@ def validate_entity_labels(cls, value: list[str] | None) -> list[str] | None:
logger.warning("entity_labels contained duplicates, removed automatically.")
return deduped
+ @field_validator("entity_label_denylist")
+ @classmethod
+ def validate_entity_label_denylist(cls, value: list[str] | None) -> list[str] | None:
+ if value is None:
+ return value
+ cleaned = [label.strip().lower() for label in value if label.strip()]
+ if not cleaned:
+ raise ValueError("entity_label_denylist must not be empty. Use None to disable the deny list.")
+ deduped = sorted(set(cleaned))
+ if len(deduped) != len(cleaned):
+ logger.warning("entity_label_denylist contained duplicates, removed automatically.")
+ return deduped
+
+ @model_validator(mode="after")
+ def warn_on_allowlist_denylist_overlap(self) -> "Detect":
+ if self.entity_labels is not None and self.entity_label_denylist is not None:
+ overlap = sorted(set(self.entity_labels) & set(self.entity_label_denylist))
+ if overlap:
+ logger.warning(
+ "entity_labels and entity_label_denylist share labels that will never be detected: %s",
+ overlap,
+ )
+ return self
+
class Rewrite(BaseModel):
"""Configuration for rewrite-mode execution."""
diff --git a/src/anonymizer/engine/detection/detection_workflow.py b/src/anonymizer/engine/detection/detection_workflow.py
index a577a47b..a92253a7 100644
--- a/src/anonymizer/engine/detection/detection_workflow.py
+++ b/src/anonymizer/engine/detection/detection_workflow.py
@@ -94,6 +94,7 @@ def detect_and_validate_entities(
validation_excerpt_window_chars: int = _DEFAULT_VALIDATION_EXCERPT_WINDOW_CHARS,
validation_single_chunk_full_text: bool = True,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
data_summary: str | None = None,
preview_num_records: int | None = None,
) -> EntityDetectionResult:
@@ -113,6 +114,7 @@ def detect_and_validate_entities(
validation_excerpt_window_chars=validation_excerpt_window_chars,
validation_single_chunk_full_text=validation_single_chunk_full_text,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
detection_result = self._adapter.run_workflow(
@@ -135,6 +137,7 @@ def _build_detection_spec(
validation_excerpt_window_chars: int = _DEFAULT_VALIDATION_EXCERPT_WINDOW_CHARS,
validation_single_chunk_full_text: bool = True,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
data_summary: str | None = None,
) -> tuple[list[ModelConfig], list[ColumnConfigT]]:
"""Build the (model_configs, columns) for the core detection workflow.
@@ -143,7 +146,7 @@ def _build_detection_spec(
and :meth:`build_detection_config` (which exports it for an external runtime),
so both paths run exactly the same workflow.
"""
- labels = _resolve_detection_labels(entity_labels)
+ labels = _resolve_detection_labels(entity_labels, set(entity_label_denylist) if entity_label_denylist else None)
workflow_model_configs = self._inject_detector_params(
model_configs=model_configs,
selected_models=selected_models,
@@ -240,6 +243,7 @@ def build_detection_config(
validation_excerpt_window_chars: int = _DEFAULT_VALIDATION_EXCERPT_WINDOW_CHARS,
validation_single_chunk_full_text: bool = True,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
data_summary: str | None = None,
) -> DataDesignerConfigBuilder:
"""Build (without executing) the core detection workflow as a DataDesigner
@@ -255,6 +259,7 @@ def build_detection_config(
validation_excerpt_window_chars=validation_excerpt_window_chars,
validation_single_chunk_full_text=validation_single_chunk_full_text,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
return self._adapter.build_config(
@@ -275,6 +280,7 @@ def build_detection_builder_for_seed(
validation_excerpt_window_chars: int = _DEFAULT_VALIDATION_EXCERPT_WINDOW_CHARS,
validation_single_chunk_full_text: bool = True,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
data_summary: str | None = None,
job_index: int = 0,
num_jobs: int = 1,
@@ -295,6 +301,7 @@ def build_detection_builder_for_seed(
validation_excerpt_window_chars=validation_excerpt_window_chars,
validation_single_chunk_full_text=validation_single_chunk_full_text,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
return self._adapter.build_config_for_seed(
@@ -313,6 +320,7 @@ def identify_latent_entities(
selected_models: DetectionModelSelection,
gliner_detection_threshold: float,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
privacy_goal: PrivacyGoal | None,
data_summary: str | None = None,
preview_num_records: int | None = None,
@@ -322,7 +330,7 @@ def identify_latent_entities(
Runs after ``detect_and_validate_entities`` when rewrite mode is
enabled. Uses an LLM to identify entities inferable from context.
"""
- labels = _resolve_detection_labels(entity_labels)
+ labels = _resolve_detection_labels(entity_labels, set(entity_label_denylist) if entity_label_denylist else None)
workflow_model_configs = self._inject_detector_params(
model_configs=model_configs,
selected_models=selected_models,
@@ -339,6 +347,7 @@ def identify_latent_entities(
prompt=_get_latent_prompt(
data_summary=data_summary,
privacy_goal=privacy_goal,
+ entity_label_denylist=entity_label_denylist,
),
model_alias=latent_alias,
output_format=LatentEntitiesSchema,
@@ -347,7 +356,12 @@ def identify_latent_entities(
workflow_name="latent-entity-detection",
preview_num_records=preview_num_records,
)
- return EntityDetectionResult(dataframe=latent_result.dataframe, failed_records=latent_result.failed_records)
+ latent_df = latent_result.dataframe.copy()
+ if COL_LATENT_ENTITIES in latent_df.columns:
+ latent_df[COL_LATENT_ENTITIES] = latent_df[COL_LATENT_ENTITIES].apply(
+ lambda raw: _filter_denied_latent_entities(raw, entity_label_denylist)
+ )
+ return EntityDetectionResult(dataframe=latent_df, failed_records=latent_result.failed_records)
def run(
self,
@@ -360,6 +374,7 @@ def run(
validation_excerpt_window_chars: int = _DEFAULT_VALIDATION_EXCERPT_WINDOW_CHARS,
validation_single_chunk_full_text: bool = True,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
privacy_goal: PrivacyGoal | None = None,
data_summary: str | None = None,
tag_latent_entities: bool = True,
@@ -390,6 +405,7 @@ def run(
validation_excerpt_window_chars=validation_excerpt_window_chars,
validation_single_chunk_full_text=validation_single_chunk_full_text,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
preview_num_records=preview_num_records,
)
@@ -401,6 +417,7 @@ def run(
selected_models=selected_models,
gliner_detection_threshold=gliner_detection_threshold,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
privacy_goal=privacy_goal,
data_summary=data_summary,
preview_num_records=preview_num_records,
@@ -417,8 +434,11 @@ def run(
# TODO(docs): document this None-vs-explicit contract in user-facing docs.
if COL_DETECTED_ENTITIES in final_df.columns:
allowed = set(entity_labels) if entity_labels is not None else None
+ entity_label_denylist_set = set(entity_label_denylist) if entity_label_denylist else None
final_df[COL_FINAL_ENTITIES] = final_df[COL_DETECTED_ENTITIES].apply(
- lambda raw: _materialize_final_entities(raw, allowed_labels=allowed)
+ lambda raw: _materialize_final_entities(
+ raw, allowed_labels=allowed, entity_label_denylist=entity_label_denylist_set
+ )
)
if compute_grouped:
final_df[COL_ENTITIES_BY_VALUE] = final_df[COL_FINAL_ENTITIES].apply(_build_entities_by_value)
@@ -455,21 +475,73 @@ def _inject_detector_params(
return resolved
-def _resolve_detection_labels(entity_labels: list[str] | None) -> list[str]:
- if entity_labels is None:
- return list(DEFAULT_ENTITY_LABELS)
- return list(entity_labels)
+def _resolve_detection_labels(
+ entity_labels: list[str] | None,
+ entity_label_denylist: set[str] | None = None,
+) -> list[str]:
+ labels = list(DEFAULT_ENTITY_LABELS) if entity_labels is None else list(entity_labels)
+ if entity_label_denylist:
+ denied = {label.casefold() for label in entity_label_denylist}
+ labels = [label for label in labels if label.casefold() not in denied]
+ if not labels:
+ logger.warning(
+ "entity_label_denylist removed all labels from the effective detection set. No entities will be detected."
+ )
+ return labels
-def _materialize_final_entities(raw: object, *, allowed_labels: set[str] | None) -> dict:
- """Build COL_FINAL_ENTITIES, optionally filtering to *allowed_labels*."""
+def _materialize_final_entities(
+ raw: object,
+ *,
+ allowed_labels: set[str] | None,
+ entity_label_denylist: set[str] | None,
+) -> dict:
+ """Build COL_FINAL_ENTITIES, optionally filtering to *allowed_labels* and excluding *entity_label_denylist*."""
parsed = EntitiesSchema.from_raw(raw)
- if allowed_labels is None:
- return parsed.model_dump()
- kept = [e for e in parsed.entities if e.label in allowed_labels]
+ allowed = {label.casefold() for label in allowed_labels} if allowed_labels is not None else None
+ denied = {label.casefold() for label in entity_label_denylist or []}
+ kept = [
+ e
+ for e in parsed.entities
+ if (allowed is None or e.label.strip().casefold() in allowed) and e.label.strip().casefold() not in denied
+ ]
return EntitiesSchema(entities=kept).model_dump()
+def _filter_denied_latent_entities(raw: object, entity_label_denylist: list[str] | None) -> object:
+ """Remove denied latent labels while preserving the structured payload shape."""
+ denied = {label.casefold() for label in entity_label_denylist or []}
+ if not denied:
+ return raw
+
+ if isinstance(raw, LatentEntitiesSchema):
+ kept = [entity for entity in raw.latent_entities if entity.label.strip().casefold() not in denied]
+ return LatentEntitiesSchema(latent_entities=kept).model_dump(mode="json")
+
+ if isinstance(raw, dict):
+ entities = raw.get("latent_entities")
+ if not isinstance(entities, list):
+ return raw
+ return {
+ **raw,
+ "latent_entities": [
+ entity
+ for entity in entities
+ if not isinstance(entity, dict) or str(entity.get("label", "")).strip().casefold() not in denied
+ ],
+ }
+
+ # Retain support for legacy list-shaped traces.
+ if isinstance(raw, list):
+ return [
+ entity
+ for entity in raw
+ if not isinstance(entity, dict) or str(entity.get("label", "")).strip().casefold() not in denied
+ ]
+
+ return raw
+
+
def _build_entities_by_value(final_entities_raw: object) -> dict:
"""Derive COL_ENTITIES_BY_VALUE from COL_FINAL_ENTITIES."""
parsed = EntitiesSchema.from_raw(final_entities_raw)
@@ -695,12 +767,26 @@ def _get_augment_prompt(*, data_summary: str | None, labels: list[str], strict_l
)
-def _get_latent_prompt(*, data_summary: str | None, privacy_goal: PrivacyGoal | None) -> str:
+def _get_latent_prompt(
+ *,
+ data_summary: str | None,
+ privacy_goal: PrivacyGoal | None,
+ entity_label_denylist: list[str] | None = None,
+) -> str:
summary_line = data_summary.strip() if data_summary else "Not provided"
privacy_goal_text = _format_privacy_goal(privacy_goal)
+ denied_labels = sorted({label.strip().casefold() for label in entity_label_denylist or [] if label.strip()})
+ denylist_block = (
+ "\n\n"
+ f"Do NOT return latent entities with these labels: {', '.join(denied_labels)}.\n"
+ "\n"
+ if denied_labels
+ else ""
+ )
prompt = """You are performing: LATENT ENTITY & INFERENCE ANALYSIS for privacy protection.
The text will be rewritten according to this privacy goal: <>
+<>
Goal: Identify sensitive information that is NOT explicitly stated in the text, \
but is reasonably inferable from context and could materially increase re-identification \
@@ -788,6 +874,7 @@ def _get_latent_prompt(*, data_summary: str | None, privacy_goal: PrivacyGoal |
"<>": privacy_goal_text,
"<>": summary_line,
"<>": _jinja(COL_TAGGED_TEXT),
+ "<>": denylist_block,
},
)
diff --git a/src/anonymizer/engine/evaluation/entity_coverage_judge.py b/src/anonymizer/engine/evaluation/entity_coverage_judge.py
index a260a122..292427fc 100644
--- a/src/anonymizer/engine/evaluation/entity_coverage_judge.py
+++ b/src/anonymizer/engine/evaluation/entity_coverage_judge.py
@@ -68,9 +68,37 @@ class EntityCoverageSchema(BaseModel):
# ---------------------------------------------------------------------------
-def _entity_type_scope_block(entity_labels: list[str] | None) -> str:
+def _effective_entity_labels(
+ entity_labels: list[str] | None,
+ entity_label_denylist: list[str] | None,
+) -> list[str] | None:
+ """Return the effective allowlist for prompt and filter scope.
+
+ ``None`` remains permissive so coverage includes novel labels introduced by
+ augmentation. Denied labels are applied independently by the prompt and
+ postprocessing filter.
+ """
+ if entity_labels is None:
+ return None
+ if not entity_label_denylist:
+ return entity_labels
+ denied = {label.casefold() for label in entity_label_denylist}
+ effective = [label for label in entity_labels if label.casefold() not in denied]
+ return effective
+
+
+def _entity_type_scope_block(
+ entity_labels: list[str] | None,
+ entity_label_denylist: list[str] | None = None,
+) -> str:
if entity_labels is None:
- return "\nEvaluate for all PII and sensitive entity types.\n"
+ denied = sorted({label.strip().casefold() for label in entity_label_denylist or [] if label.strip()})
+ exclusion = (
+ f"\nDo NOT report candidates with these explicitly excluded entity labels: {', '.join(denied)}."
+ if denied
+ else ""
+ )
+ return f"\nEvaluate for all PII and sensitive entity types.{exclusion}\n"
labels_str = ", ".join(entity_labels)
return (
"\n"
@@ -94,18 +122,39 @@ def _data_summary_block(data_summary: str | None) -> str:
)
-def _coverage_prompt(*, entity_labels: list[str] | None, data_summary: str | None = None) -> str:
- entity_scope_block = _entity_type_scope_block(entity_labels)
+def _coverage_prompt(
+ *,
+ entity_labels: list[str] | None,
+ entity_label_denylist: list[str] | None = None,
+ data_summary: str | None = None,
+) -> str:
+ entity_scope_block = _entity_type_scope_block(entity_labels, entity_label_denylist)
data_context_section = f"\n\n{_data_summary_block(data_summary)}" if data_summary and data_summary.strip() else ""
- entity_scope_guidance = (
- "- Respect the entity_type_scope: do not report candidate values outside the configured scope."
- if entity_labels is not None
- else ""
- )
-
active_labels = entity_labels if entity_labels is not None else DEFAULT_ENTITY_LABELS
labels_str = ", ".join(active_labels)
+ if entity_labels is None:
+ taxonomy_guidance = (
+ f"Common entity types include: {labels_str}. This list is not exhaustive. "
+ "Also report other direct or quasi-identifier types supported by the original-text context."
+ )
+ label_interpretation = (
+ "Use a concise snake_case label that describes each candidate's semantic entity category. "
+ "Do not use an explicitly excluded label."
+ )
+ entity_scope_guidance = (
+ "- Respect the entity_type_scope: report any supported sensitive entity type except explicit exclusions."
+ )
+ else:
+ taxonomy_guidance = f"These entity types are in scope: {labels_str}."
+ label_interpretation = (
+ "Treat each configured label as a semantic entity category. Labels may use compact, compound, "
+ "or abbreviated names; interpret their intended meaning from the label and the original-text "
+ "context. Return labels exactly as they appear in the entity_type_scope."
+ )
+ entity_scope_guidance = (
+ "- Respect the entity_type_scope: do not report candidate values outside the configured scope."
+ )
prompt = f"""You are a privacy-entity span extractor. Your task is defined below.
@@ -121,7 +170,7 @@ def _coverage_prompt(*, entity_labels: list[str] | None, data_summary: str | Non
-These entity types are in scope: {labels_str}.
+{taxonomy_guidance}
Quasi-identifiers: combinations of values that together re-identify someone \
(e.g. job title + employer + city appearing together). Time values (specific timestamps, \
times of day, schedules) can act as quasi-identifiers when combined with other attributes \
@@ -131,9 +180,7 @@ def _coverage_prompt(*, entity_labels: list[str] | None, data_summary: str | Non
{entity_scope_block}
-Treat each configured label as a semantic entity category. Labels may use compact, compound, \
-or abbreviated names; interpret their intended meaning from the label and the original-text \
-context. Return labels exactly as they appear in the entity_type_scope.
+{label_interpretation}
@@ -168,7 +215,7 @@ def _coverage_prompt(*, entity_labels: list[str] | None, data_summary: str | Non
Do flag:
- `reasoning` MUST be one sentence explaining which in-scope semantic type the value represents.
-- A value that fills the role of a listed sensitive type in context, even when it is
+- A value that fills the role of an in-scope sensitive type in context, even when it is
short, a single token, an unfamiliar or foreign-looking word, or resembles an ordinary
word or number. Decide by the value's role in the surrounding text, not by its length,
rarity, or familiarity. (This still excludes pronouns and generic references that only
@@ -345,12 +392,14 @@ def _normalize_literal_text(value: object) -> str:
def _filter_out_of_scope_entities(
entities: list[_CandidateT],
entity_labels: list[str] | None,
+ entity_label_denylist: list[str] | None = None,
) -> list[_CandidateT]:
"""Drop entities with empty labels or labels outside the configured scope.
When ``entity_labels`` is None all labels are in scope; only empty labels
- are dropped. This mirrors the prompt's scope instruction deterministically
- so a model that returns out-of-scope labels does not lower the coverage score.
+ and explicitly denied labels are dropped. This mirrors the prompt's scope
+ instruction deterministically so a model that returns out-of-scope labels
+ does not lower the coverage score.
Label drift (e.g. the model returning ``"given_name"`` instead of
``"first_name"``) is unlikely in practice — the prompt explicitly instructs
@@ -360,12 +409,14 @@ def _filter_out_of_scope_entities(
meaningfully risking false negatives on well-formed responses.
"""
allowed = {label.casefold() for label in entity_labels} if entity_labels is not None else None
+ denied = {label.casefold() for label in entity_label_denylist or []}
result = []
for entity in entities:
label = str(entity.get("label", "")).strip()
if not label:
continue
- if allowed is not None and label.casefold() not in allowed:
+ normalized_label = label.casefold()
+ if (allowed is not None and normalized_label not in allowed) or normalized_label in denied:
continue
result.append(entity)
return result
@@ -408,6 +459,11 @@ class EntityCoverageWorkflow(_BaseJudgeWorkflow):
The judge independently extracts candidates from the original text and entity-type
scope. Deterministic postprocessing removes nonliteral and already-covered findings.
+ ``entity_labels`` scopes evaluation to a specific allowlist of labels (``None`` means
+ all labels). ``entity_label_denylist`` further excludes specific labels from
+ scope regardless of ``entity_labels``. Both are applied to the LLM prompt and the
+ postprocess filter so denied labels are never penalised in the coverage score.
+
Output columns:
``COL_ENTITY_COVERAGE`` (float|None) — covered / total unique candidate values
``COL_MISSED_ENTITIES`` (list) — missed entities with value, label, reasoning
@@ -428,10 +484,12 @@ def __init__(
adapter: NddAdapter,
*,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
data_summary: str | None = None,
) -> None:
super().__init__(adapter)
self._entity_labels = entity_labels
+ self._entity_label_denylist = entity_label_denylist
self._data_summary = data_summary
# ------------------------------------------------------------------ hooks
@@ -466,10 +524,12 @@ def _extract_invalid(cls, parsed: BaseModel) -> list[dict[str, object]]:
def column_config(self, selected_models: EvaluateModelSelection) -> LLMStructuredColumnConfig:
"""Override to inject instance-specific entity_labels and data_summary."""
+ effective_labels = _effective_entity_labels(self._entity_labels, self._entity_label_denylist)
return LLMStructuredColumnConfig(
name=self.RAW_COL,
prompt=_coverage_prompt(
- entity_labels=self._entity_labels,
+ entity_labels=effective_labels,
+ entity_label_denylist=self._entity_label_denylist,
data_summary=self._data_summary,
),
model_alias=resolve_model_alias(self.MODEL_ROLE, selected_models),
@@ -492,7 +552,11 @@ def postprocess(self, dataframe: pd.DataFrame) -> pd.DataFrame:
missed_entities_list.append([])
n_candidates_list.append(None)
else:
- candidates = _filter_out_of_scope_entities(candidates, self._entity_labels)
+ candidates = _filter_out_of_scope_entities(
+ candidates,
+ _effective_entity_labels(self._entity_labels, self._entity_label_denylist),
+ self._entity_label_denylist,
+ )
candidates = _filter_nonliteral_entities(candidates, out[COL_TEXT].loc[idx])
candidates = _deduplicate_candidate_values(candidates)
n_candidates = len(candidates)
diff --git a/src/anonymizer/engine/replace/replace_runner.py b/src/anonymizer/engine/replace/replace_runner.py
index 8da79241..61b8941d 100644
--- a/src/anonymizer/engine/replace/replace_runner.py
+++ b/src/anonymizer/engine/replace/replace_runner.py
@@ -119,6 +119,7 @@ def evaluate(
selected_models: EvaluateModelSelection,
preview_num_records: int | None = None,
entity_labels: list[str] | None = None,
+ entity_label_denylist: list[str] | None = None,
compute_detection_validity: bool = False,
data_summary: str | None = None,
) -> ReplacementResult:
@@ -129,6 +130,10 @@ def evaluate(
Detection validity runs only when ``compute_detection_validity=True``.
All active judges are submitted as columns of one DataDesigner workflow.
+ ``entity_labels`` and ``entity_label_denylist`` together define the label
+ scope passed to the coverage judge — only entities whose labels were in
+ scope during detection are evaluated.
+
Raises ``ValueError`` if the workflow has no adapter wired up or if the
dataframe is missing the columns the judges read.
"""
@@ -151,6 +156,7 @@ def evaluate(
entity_coverage_judge = EntityCoverageWorkflow(
adapter=self._adapter, # type: ignore[arg-type]
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
failed_records: list[FailedRecord] = []
diff --git a/src/anonymizer/interface/anonymizer.py b/src/anonymizer/interface/anonymizer.py
index d7c26eba..8255ee32 100644
--- a/src/anonymizer/interface/anonymizer.py
+++ b/src/anonymizer/interface/anonymizer.py
@@ -302,6 +302,7 @@ def export_detection_config(
validation_max_entities_per_call=config.detect.validation_max_entities_per_call,
validation_excerpt_window_chars=config.detect.validation_excerpt_window_chars,
entity_labels=config.detect.entity_labels,
+ entity_label_denylist=config.detect.entity_label_denylist,
data_summary=data.data_summary,
)
@@ -334,6 +335,7 @@ def export_detection_builder_for_seed(
validation_max_entities_per_call=config.detect.validation_max_entities_per_call,
validation_excerpt_window_chars=config.detect.validation_excerpt_window_chars,
entity_labels=config.detect.entity_labels,
+ entity_label_denylist=config.detect.entity_label_denylist,
data_summary=data_summary,
job_index=job_index,
num_jobs=num_jobs,
@@ -373,6 +375,7 @@ def preview(
replace_method=config.replace,
rewrite_config=config.rewrite.privacy_goal if config.rewrite is not None else None,
entity_labels=config.detect.entity_labels,
+ entity_label_denylist=config.detect.entity_label_denylist,
data_summary=result.data_summary,
)
except KeyboardInterrupt:
@@ -455,6 +458,7 @@ def evaluate(
raise InvalidConfigError(str(exc)) from exc
entity_labels = getattr(output, "entity_labels", None)
+ entity_label_denylist = getattr(output, "entity_label_denylist", None)
data_summary = getattr(output, "data_summary", None)
num_records = len(output.trace_dataframe)
mode_name = "rewrite" if is_rewrite else type(replace_method).__name__
@@ -518,6 +522,7 @@ def evaluate(
coverage_wf = EntityCoverageWorkflow(
adapter=self._adapter,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
logger.info(LOG_INDENT + "🔎 Running entity coverage")
@@ -549,6 +554,7 @@ def evaluate(
failed_records=all_failed,
rewrite_config=rewrite_config,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
else:
@@ -562,6 +568,7 @@ def evaluate(
model_configs=self._model_configs,
selected_models=self._selected_models.evaluate,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
compute_detection_validity=evaluate_config.compute_detection_validity,
data_summary=data_summary,
)
@@ -597,6 +604,7 @@ def evaluate(
failed_records=replace_result.failed_records,
replace_method=replace_method,
entity_labels=entity_labels,
+ entity_label_denylist=entity_label_denylist,
data_summary=data_summary,
)
@@ -711,6 +719,7 @@ def _run_internal_impl(
validation_max_entities_per_call=config.detect.validation_max_entities_per_call,
validation_excerpt_window_chars=config.detect.validation_excerpt_window_chars,
entity_labels=config.detect.entity_labels,
+ entity_label_denylist=config.detect.entity_label_denylist,
privacy_goal=config.rewrite.privacy_goal if config.rewrite else None,
data_summary=data.data_summary,
tag_latent_entities=config.rewrite is not None,
@@ -803,6 +812,7 @@ def _run_internal_impl(
replace_method=config.replace,
rewrite_config=config.rewrite.privacy_goal if config.rewrite is not None else None,
entity_labels=config.detect.entity_labels,
+ entity_label_denylist=config.detect.entity_label_denylist,
data_summary=data.data_summary,
)
diff --git a/src/anonymizer/interface/results.py b/src/anonymizer/interface/results.py
index 7bfe13f0..8f82791e 100644
--- a/src/anonymizer/interface/results.py
+++ b/src/anonymizer/interface/results.py
@@ -64,6 +64,13 @@ class AnonymizerResult(_DisplayMixin):
mode was used. Set by ``run()`` / ``preview()``; consumed by
``evaluate()`` to dispatch the rewrite judges. Mutually exclusive
with ``replace_method``.
+ entity_labels: Allowlist of entity labels that were in scope during
+ detection. Preserved for ``evaluate()`` so the coverage judge scopes
+ its evaluation to the same label set. ``None`` means all default
+ labels were in scope.
+ entity_label_denylist: Labels that were explicitly excluded from
+ detection. Preserved for ``evaluate()`` so the coverage judge does
+ not penalise the output for not anonymizing denied labels.
data_summary: Optional dataset context supplied with the original input.
Preserved for ``evaluate()`` so entity-coverage judging uses the
same context as detection.
@@ -76,6 +83,7 @@ class AnonymizerResult(_DisplayMixin):
replace_method: ReplaceMethod | None = None
rewrite_config: PrivacyGoal | None = None
entity_labels: list[str] | None = None
+ entity_label_denylist: list[str] | None = None
data_summary: str | None = None
_display_cycle_index: int = field(default=0, init=False, repr=False)
@@ -110,6 +118,13 @@ class PreviewResult(_DisplayMixin):
rewrite_config: The privacy goal that produced this preview when rewrite
mode was used. Set by ``preview()``; consumed by ``evaluate()`` to
dispatch the rewrite judges. Mutually exclusive with ``replace_method``.
+ entity_labels: Allowlist of entity labels that were in scope during
+ detection. Preserved for ``evaluate()`` so the coverage judge scopes
+ its evaluation to the same label set. ``None`` means all default
+ labels were in scope.
+ entity_label_denylist: Labels that were explicitly excluded from
+ detection. Preserved for ``evaluate()`` so the coverage judge does
+ not penalise the output for not anonymizing denied labels.
data_summary: Optional dataset context supplied with the original input.
Preserved for ``evaluate()`` so entity-coverage judging uses the
same context as detection.
@@ -123,6 +138,7 @@ class PreviewResult(_DisplayMixin):
replace_method: ReplaceMethod | None = None
rewrite_config: PrivacyGoal | None = None
entity_labels: list[str] | None = None
+ entity_label_denylist: list[str] | None = None
data_summary: str | None = None
_display_cycle_index: int = field(default=0, init=False, repr=False)
diff --git a/src/anonymizer/measurement/records/run.py b/src/anonymizer/measurement/records/run.py
index 4a9e95b9..fcff24d9 100644
--- a/src/anonymizer/measurement/records/run.py
+++ b/src/anonymizer/measurement/records/run.py
@@ -20,11 +20,13 @@ def _detect_config_metadata(detect: Any | None) -> dict[str, Any]:
entity_label_count = len(DEFAULT_ENTITY_LABELS)
else:
entity_label_count = len(entity_labels)
+ entity_label_denylist = getattr(detect, "entity_label_denylist", None)
return {
"gliner_threshold": getattr(detect, "gliner_threshold", None),
"entity_label_source": "custom" if entity_labels is not None else "default",
"entity_label_count": entity_label_count,
"entity_labels": list(entity_labels) if entity_labels is not None else None,
+ "entity_label_denylist": list(entity_label_denylist) if entity_label_denylist is not None else None,
"validation_max_entities_per_call": getattr(detect, "validation_max_entities_per_call", None),
"validation_excerpt_window_chars": getattr(detect, "validation_excerpt_window_chars", None),
}
diff --git a/tests/config/test_anonymizer_config.py b/tests/config/test_anonymizer_config.py
index 0738208c..02e7daa8 100644
--- a/tests/config/test_anonymizer_config.py
+++ b/tests/config/test_anonymizer_config.py
@@ -3,12 +3,18 @@
from __future__ import annotations
+import logging
from pathlib import Path
import pytest
from pydantic import ValidationError
-from anonymizer.config.anonymizer_config import AnonymizerConfig, AnonymizerInput, Rewrite, infer_input_source_suffix
+from anonymizer.config.anonymizer_config import (
+ AnonymizerConfig,
+ AnonymizerInput,
+ Rewrite,
+ infer_input_source_suffix,
+)
from anonymizer.config.replace_strategies import (
Annotate,
Hash,
@@ -147,3 +153,72 @@ def test_detect_validation_max_entities_per_call_must_be_positive() -> None:
def test_detect_validation_excerpt_window_chars_must_be_positive() -> None:
with pytest.raises(ValidationError):
AnonymizerConfig(detect={"validation_excerpt_window_chars": 0}, replace=Redact())
+
+
+# ── entity_label_denylist ─────────────────────────────────────────────────────
+
+
+def test_entity_label_denylist_defaults_to_none() -> None:
+ config = AnonymizerConfig(replace=Redact())
+ assert config.detect.entity_label_denylist is None
+
+
+def test_entity_label_denylist_accepts_list() -> None:
+ config = AnonymizerConfig(detect={"entity_label_denylist": ["EMAIL", "city"]}, replace=Redact())
+ assert config.detect.entity_label_denylist is not None
+ assert set(config.detect.entity_label_denylist) == {"email", "city"}
+
+
+def test_entity_label_denylist_strips_whitespace_and_lowercases() -> None:
+ config = AnonymizerConfig(detect={"entity_label_denylist": [" FIRST_NAME ", "Email"]}, replace=Redact())
+ assert config.detect.entity_label_denylist is not None
+ assert "first_name" in config.detect.entity_label_denylist
+ assert "email" in config.detect.entity_label_denylist
+
+
+def test_entity_label_denylist_deduplicates(caplog: pytest.LogCaptureFixture) -> None:
+ with caplog.at_level(logging.WARNING, logger="anonymizer"):
+ config = AnonymizerConfig(detect={"entity_label_denylist": ["email", "email"]}, replace=Redact())
+ assert config.detect.entity_label_denylist == ["email"]
+ assert "duplicates" in caplog.text
+
+
+def test_entity_label_denylist_empty_list_raises() -> None:
+ with pytest.raises(ValidationError, match="must not be empty"):
+ AnonymizerConfig(detect={"entity_label_denylist": []}, replace=Redact())
+
+
+def test_entity_label_denylist_whitespace_only_raises() -> None:
+ with pytest.raises(ValidationError, match="must not be empty"):
+ AnonymizerConfig(detect={"entity_label_denylist": [" ", ""]}, replace=Redact())
+
+
+def test_entity_label_denylist_overlap_with_entity_labels_warns(caplog: pytest.LogCaptureFixture) -> None:
+ with caplog.at_level(logging.WARNING, logger="anonymizer"):
+ AnonymizerConfig(
+ detect={"entity_labels": ["email", "city"], "entity_label_denylist": ["email"]},
+ replace=Redact(),
+ )
+ assert "email" in caplog.text
+ assert "will never be detected" in caplog.text
+
+
+def test_entity_label_denylist_no_overlap_does_not_warn(caplog: pytest.LogCaptureFixture) -> None:
+ with caplog.at_level(logging.WARNING, logger="anonymizer"):
+ AnonymizerConfig(
+ detect={"entity_labels": ["email", "city"], "entity_label_denylist": ["first_name"]},
+ replace=Redact(),
+ )
+ assert "will never be detected" not in caplog.text
+
+
+def test_entity_label_denylist_overlap_warning_only_fires_when_allowlist_explicit(
+ caplog: pytest.LogCaptureFixture,
+) -> None:
+ """No warning when entity_labels=None (defaults) even if denylist is set."""
+ with caplog.at_level(logging.WARNING, logger="anonymizer"):
+ AnonymizerConfig(
+ detect={"entity_label_denylist": ["email"]},
+ replace=Redact(),
+ )
+ assert "will never be detected" not in caplog.text
diff --git a/tests/engine/test_detection_config_serialization.py b/tests/engine/test_detection_config_serialization.py
index 0acda66a..976c7913 100644
--- a/tests/engine/test_detection_config_serialization.py
+++ b/tests/engine/test_detection_config_serialization.py
@@ -93,6 +93,59 @@ def test_detection_builder_round_trips_through_native_data_designer_config(tmp_p
assert "generator_params" not in serialized_text
+def _get_gliner_labels_from_builder(builder: DataDesignerConfigBuilder) -> list[str]:
+ payload = builder.get_builder_config().to_json()
+ assert payload is not None
+ serialized = json.loads(payload)
+ model_configs = serialized["data_designer"]["model_configs"]
+ gliner = next(m for m in model_configs if m.get("alias") == "gliner-pii-detector")
+ return gliner["inference_parameters"]["extra_body"]["labels"]
+
+
+def test_build_detection_builder_for_seed_respects_entity_label_denylist(tmp_path: Path) -> None:
+ seed_path = tmp_path / "seed.parquet"
+ pd.DataFrame({COL_TEXT: ["Alice"]}).to_parquet(seed_path, index=False)
+
+ parsed_models = parse_model_configs(None)
+ workflow = EntityDetectionWorkflow(adapter=NddAdapter(data_designer=cast(DataDesigner, Mock())))
+ builder = workflow.build_detection_builder_for_seed(
+ seed_path=seed_path,
+ model_configs=parsed_models.model_configs,
+ selected_models=parsed_models.selected_models.detection,
+ gliner_detection_threshold=0.3,
+ entity_labels=["first_name", "email", "city"],
+ entity_label_denylist=["email"],
+ )
+
+ labels = _get_gliner_labels_from_builder(builder)
+ assert "email" not in labels
+ assert "first_name" in labels
+ assert "city" in labels
+
+
+def test_build_detection_config_respects_entity_label_denylist(tmp_path: Path) -> None:
+ seed_path = tmp_path / "seed.parquet"
+ input_df = pd.DataFrame({COL_TEXT: ["Alice"]})
+ input_df.to_parquet(seed_path, index=False)
+
+ parsed_models = parse_model_configs(None)
+ workflow = EntityDetectionWorkflow(adapter=NddAdapter(data_designer=cast(DataDesigner, Mock())))
+ builder = workflow.build_detection_config(
+ input_df,
+ seed_path=seed_path,
+ model_configs=parsed_models.model_configs,
+ selected_models=parsed_models.selected_models.detection,
+ gliner_detection_threshold=0.3,
+ entity_labels=["first_name", "email", "city"],
+ entity_label_denylist=["email"],
+ )
+
+ labels = _get_gliner_labels_from_builder(builder)
+ assert "email" not in labels
+ assert "first_name" in labels
+ assert "city" in labels
+
+
def test_fresh_process_discovers_plugins_when_loading_native_config(tmp_path: Path) -> None:
seed_path = tmp_path / "seed.parquet"
pd.DataFrame({COL_TEXT: ["Alice"]}).to_parquet(seed_path, index=False)
diff --git a/tests/engine/test_detection_workflow.py b/tests/engine/test_detection_workflow.py
index aed0928a..380b6440 100644
--- a/tests/engine/test_detection_workflow.py
+++ b/tests/engine/test_detection_workflow.py
@@ -4,6 +4,7 @@
from __future__ import annotations
import json
+import logging
from unittest.mock import Mock
import pandas as pd
@@ -33,10 +34,12 @@
)
from anonymizer.engine.detection.detection_workflow import (
EntityDetectionWorkflow,
+ _filter_denied_latent_entities,
_format_label_examples,
_get_augment_prompt,
_get_latent_prompt,
_get_validation_prompt,
+ _materialize_final_entities,
_resolve_detection_labels,
)
from anonymizer.engine.ndd.adapter import FailedRecord, WorkflowRunResult
@@ -157,6 +160,66 @@ def test_latent_prompt_includes_summary_and_goal() -> None:
assert COL_TAGGED_TEXT in prompt
+def test_latent_prompt_excludes_denied_labels() -> None:
+ prompt = _get_latent_prompt(
+ data_summary=None,
+ privacy_goal=None,
+ entity_label_denylist=["Health_Condition", "occupation"],
+ )
+ assert "Do NOT return latent entities with these labels: health_condition, occupation." in prompt
+
+
+def test_filter_denied_latent_entities_is_case_insensitive() -> None:
+ raw = {
+ "latent_entities": [
+ {"label": "Health_Condition", "value": "diabetes"},
+ {"label": "employer", "value": "Acme"},
+ ]
+ }
+ result = _filter_denied_latent_entities(raw, ["health_condition"])
+ assert result == {"latent_entities": [{"label": "employer", "value": "Acme"}]}
+
+
+def test_identify_latent_entities_filters_denied_labels(
+ stub_detector_model_configs: list[ModelConfig],
+ stub_detection_model_selection: DetectionModelSelection,
+) -> None:
+ adapter = Mock()
+ adapter.run_workflow.return_value = WorkflowRunResult(
+ dataframe=pd.DataFrame(
+ {
+ COL_TEXT: ["The patient works at Acme."],
+ COL_LATENT_ENTITIES: [
+ {
+ "latent_entities": [
+ {"label": "Health_Condition", "value": "diabetes"},
+ {"label": "employer", "value": "Acme"},
+ ]
+ }
+ ],
+ }
+ ),
+ failed_records=[],
+ )
+ workflow = EntityDetectionWorkflow(adapter=adapter)
+
+ result = workflow.identify_latent_entities(
+ pd.DataFrame({COL_TEXT: ["The patient works at Acme."]}),
+ model_configs=stub_detector_model_configs,
+ selected_models=stub_detection_model_selection,
+ gliner_detection_threshold=0.5,
+ entity_label_denylist=["health_condition"],
+ privacy_goal=PrivacyGoal(
+ protect="Protect inferred sensitive attributes.",
+ preserve="Preserve non-sensitive facts.",
+ ),
+ )
+
+ assert result.dataframe[COL_LATENT_ENTITIES].iloc[0] == {
+ "latent_entities": [{"label": "employer", "value": "Acme"}]
+ }
+
+
def test_run_without_latent_detection_materializes_final_entities(
stub_detector_model_configs: list[ModelConfig],
stub_detection_model_selection: DetectionModelSelection,
@@ -462,6 +525,202 @@ def test_default_entity_labels_preserves_novel_augmented_entities(
assert "ipv4" in final_labels
+# ── entity_label_denylist ─────────────────────────────────────────────────────
+
+
+def test_resolve_detection_labels_denylist_removes_labels() -> None:
+ labels = _resolve_detection_labels(["first_name", "email", "city"], entity_label_denylist={"email"})
+ assert "email" not in labels
+ assert "first_name" in labels
+ assert "city" in labels
+
+
+def test_resolve_detection_labels_denylist_is_case_insensitive() -> None:
+ labels = _resolve_detection_labels(["first_name", "Email"], entity_label_denylist={"EMAIL"})
+ assert labels == ["first_name"]
+
+
+def test_resolve_detection_labels_denylist_on_defaults() -> None:
+ labels = _resolve_detection_labels(None, entity_label_denylist={"ssn", "first_name"})
+ assert "ssn" not in labels
+ assert "first_name" not in labels
+ assert "email" in labels
+
+
+def test_resolve_detection_labels_none_denylist_is_noop() -> None:
+ labels = _resolve_detection_labels(["email", "city"], entity_label_denylist=None)
+ assert labels == ["email", "city"]
+
+
+def test_resolve_detection_labels_empty_result_warns(caplog: pytest.LogCaptureFixture) -> None:
+ with caplog.at_level(logging.WARNING, logger="anonymizer.detection"):
+ labels = _resolve_detection_labels(["email"], entity_label_denylist={"email"})
+ assert labels == []
+ assert "No entities will be detected" in caplog.text
+
+
+def test_materialize_final_entities_applies_label_filters_case_insensitively() -> None:
+ raw = {
+ "entities": [
+ {"value": "Alice", "label": "First_Name", "start_position": 0, "end_position": 5},
+ {"value": "alice@example.com", "label": "Email", "start_position": 7, "end_position": 24},
+ {"value": "Houston", "label": "City", "start_position": 28, "end_position": 35},
+ ]
+ }
+
+ result = _materialize_final_entities(
+ raw,
+ allowed_labels={"first_name", "email"},
+ entity_label_denylist={"EMAIL"},
+ )
+
+ final = EntitiesSchema.from_raw(result)
+ assert [entity.label for entity in final.entities] == ["First_Name"]
+
+
+def test_denylist_filters_entities_from_final_entities(
+ stub_detector_model_configs: list[ModelConfig],
+ stub_detection_model_selection: DetectionModelSelection,
+) -> None:
+ adapter = Mock()
+ adapter.run_workflow.return_value = WorkflowRunResult(
+ dataframe=pd.DataFrame(
+ {
+ COL_TEXT: ["Alice works at Acme, her email is alice@example.com"],
+ COL_DETECTED_ENTITIES: [
+ {
+ "entities": [
+ {"value": "Alice", "label": "first_name", "start_position": 0, "end_position": 5},
+ {"value": "alice@example.com", "label": "email", "start_position": 33, "end_position": 50},
+ ]
+ }
+ ],
+ }
+ ),
+ failed_records=[],
+ )
+ workflow = EntityDetectionWorkflow(adapter=adapter)
+
+ result = workflow.run(
+ pd.DataFrame({COL_TEXT: ["Alice works at Acme, her email is alice@example.com"]}),
+ model_configs=stub_detector_model_configs,
+ selected_models=stub_detection_model_selection,
+ gliner_detection_threshold=0.5,
+ entity_label_denylist=["email"],
+ tag_latent_entities=False,
+ )
+
+ final = EntitiesSchema.from_raw(result.dataframe[COL_FINAL_ENTITIES].iloc[0])
+ final_labels = {e.label for e in final.entities}
+ assert "email" not in final_labels
+ assert "first_name" in final_labels
+
+
+def test_denylist_does_not_affect_col_detected_entities(
+ stub_detector_model_configs: list[ModelConfig],
+ stub_detection_model_selection: DetectionModelSelection,
+) -> None:
+ """COL_DETECTED_ENTITIES is the raw pre-filter output and must be untouched."""
+ adapter = Mock()
+ adapter.run_workflow.return_value = WorkflowRunResult(
+ dataframe=pd.DataFrame(
+ {
+ COL_TEXT: ["Alice, alice@example.com"],
+ COL_DETECTED_ENTITIES: [
+ {
+ "entities": [
+ {"value": "Alice", "label": "first_name", "start_position": 0, "end_position": 5},
+ {"value": "alice@example.com", "label": "email", "start_position": 7, "end_position": 24},
+ ]
+ }
+ ],
+ }
+ ),
+ failed_records=[],
+ )
+ workflow = EntityDetectionWorkflow(adapter=adapter)
+
+ result = workflow.run(
+ pd.DataFrame({COL_TEXT: ["Alice, alice@example.com"]}),
+ model_configs=stub_detector_model_configs,
+ selected_models=stub_detection_model_selection,
+ gliner_detection_threshold=0.5,
+ entity_label_denylist=["email"],
+ tag_latent_entities=False,
+ )
+
+ detected = EntitiesSchema.from_raw(result.dataframe[COL_DETECTED_ENTITIES].iloc[0])
+ assert "email" in {e.label for e in detected.entities}
+
+
+def test_denylist_combined_with_allowlist_allowlist_wins_for_non_denied(
+ stub_detector_model_configs: list[ModelConfig],
+ stub_detection_model_selection: DetectionModelSelection,
+) -> None:
+ """entity_labels restricts to an allowlist; denylist further removes from that set."""
+ adapter = Mock()
+ adapter.run_workflow.return_value = WorkflowRunResult(
+ dataframe=pd.DataFrame(
+ {
+ COL_TEXT: ["Alice in Houston, alice@example.com"],
+ COL_DETECTED_ENTITIES: [
+ {
+ "entities": [
+ {"value": "Alice", "label": "first_name", "start_position": 0, "end_position": 5},
+ {"value": "Houston", "label": "city", "start_position": 9, "end_position": 16},
+ {"value": "alice@example.com", "label": "email", "start_position": 18, "end_position": 35},
+ ]
+ }
+ ],
+ }
+ ),
+ failed_records=[],
+ )
+ workflow = EntityDetectionWorkflow(adapter=adapter)
+
+ result = workflow.run(
+ pd.DataFrame({COL_TEXT: ["Alice in Houston, alice@example.com"]}),
+ model_configs=stub_detector_model_configs,
+ selected_models=stub_detection_model_selection,
+ gliner_detection_threshold=0.5,
+ entity_labels=["first_name", "city", "email"],
+ entity_label_denylist=["email"],
+ tag_latent_entities=False,
+ )
+
+ final = EntitiesSchema.from_raw(result.dataframe[COL_FINAL_ENTITIES].iloc[0])
+ final_labels = {e.label for e in final.entities}
+ assert final_labels == {"first_name", "city"}
+
+
+def test_denylist_passed_to_gliner_via_labels(
+ stub_detector_model_configs: list[ModelConfig],
+ stub_detection_model_selection: DetectionModelSelection,
+) -> None:
+ """Denied labels must be absent from the label list injected into GLiNER."""
+ adapter = Mock()
+ adapter.run_workflow.return_value = WorkflowRunResult(
+ dataframe=pd.DataFrame({COL_TEXT: ["Alice"]}), failed_records=[]
+ )
+ workflow = EntityDetectionWorkflow(adapter=adapter)
+
+ workflow.run(
+ pd.DataFrame({COL_TEXT: ["Alice"]}),
+ model_configs=stub_detector_model_configs,
+ selected_models=stub_detection_model_selection,
+ gliner_detection_threshold=0.5,
+ entity_labels=["first_name", "email", "city"],
+ entity_label_denylist=["email"],
+ tag_latent_entities=False,
+ )
+
+ injected_configs = adapter.run_workflow.call_args.kwargs["model_configs"]
+ gliner_labels = injected_configs[0].inference_parameters.extra_body["labels"]
+ assert "email" not in gliner_labels
+ assert "first_name" in gliner_labels
+ assert "city" in gliner_labels
+
+
# ---------------------------------------------------------------------------
# Workflow column wiring
# ---------------------------------------------------------------------------
diff --git a/tests/engine/test_entity_coverage_judge.py b/tests/engine/test_entity_coverage_judge.py
index 7aee837e..acee219e 100644
--- a/tests/engine/test_entity_coverage_judge.py
+++ b/tests/engine/test_entity_coverage_judge.py
@@ -21,6 +21,7 @@
_FINAL_ENTITIES_FOR_COVERAGE_COL,
EntityCoverageWorkflow,
_coverage_prompt,
+ _effective_entity_labels,
_filter_out_of_scope_entities,
_find_missed_candidates,
_is_candidate_value_covered,
@@ -440,3 +441,84 @@ def test_filter_out_of_scope_entities_is_case_insensitive() -> None:
entities = [{"value": "Alice", "label": "First_Name", "reasoning": "..."}]
result = _filter_out_of_scope_entities(entities, entity_labels=["first_name"])
assert result == entities
+
+
+# ── entity_label_denylist ─────────────────────────────────────────────────────
+
+
+def test_effective_entity_labels_no_denylist_returns_entity_labels_unchanged() -> None:
+ assert _effective_entity_labels(["email", "city"], None) == ["email", "city"]
+
+
+def test_effective_entity_labels_none_labels_none_denylist_returns_none() -> None:
+ assert _effective_entity_labels(None, None) is None
+
+
+def test_effective_entity_labels_subtracts_denylist_from_explicit_labels() -> None:
+ result = _effective_entity_labels(["first_name", "email", "city"], ["email"])
+ assert result == ["first_name", "city"]
+
+
+def test_effective_entity_labels_preserves_permissive_scope_with_denylist() -> None:
+ result = _effective_entity_labels(None, ["ssn", "first_name"])
+ assert result is None
+
+
+def test_effective_entity_labels_is_case_insensitive() -> None:
+ result = _effective_entity_labels(["first_name", "Email"], ["email"])
+ assert result == ["first_name"]
+
+
+def test_coverage_prompt_excludes_denied_labels_from_scope() -> None:
+ effective = _effective_entity_labels(["first_name", "email", "city"], ["email"])
+ prompt = _coverage_prompt(entity_labels=effective)
+ assert "email" not in prompt
+ assert "first_name" in prompt
+ assert "city" in prompt
+
+
+def test_coverage_prompt_keeps_permissive_scope_and_names_denied_labels() -> None:
+ prompt = _coverage_prompt(entity_labels=None, entity_label_denylist=["email"])
+ assert "Evaluate for all PII and sensitive entity types." in prompt
+ assert "explicitly excluded entity labels: email" in prompt
+ assert "This list is not exhaustive." in prompt
+ assert "other direct or quasi-identifier types" in prompt
+ assert "Use a concise snake_case label" in prompt
+ assert "Return labels exactly as they appear" not in prompt
+
+
+def test_filter_out_of_scope_entities_keeps_novel_non_denied_labels() -> None:
+ entities = [
+ {"value": "Example Clinic", "label": "clinic_name", "reasoning": "clinic"},
+ {"value": "alice@example.com", "label": "Email", "reasoning": "email"},
+ ]
+ result = _filter_out_of_scope_entities(entities, entity_labels=None, entity_label_denylist=["email"])
+ assert result == [entities[0]]
+
+
+def test_entity_coverage_workflow_excludes_denied_labels_from_postprocess() -> None:
+ """Permissive postprocessing keeps novel labels while excluding denied labels."""
+ raw_judge_output = [
+ {"value": "Alice", "label": "first_name", "reasoning": "not replaced"},
+ {"value": "Example Clinic", "label": "clinic_name", "reasoning": "not replaced"},
+ {"value": "alice@example.com", "label": "email", "reasoning": "not replaced"},
+ ]
+ entities_by_value = {"entities_by_value": [{"value": "Alice", "labels": ["first_name"]}]}
+ input_df = pd.DataFrame(
+ {
+ COL_TEXT: ["Alice visited Example Clinic and used alice@example.com"],
+ COL_ENTITIES_BY_VALUE: [entities_by_value],
+ COL_ENTITY_COVERAGE_JUDGE: [{"candidate_entities": raw_judge_output}],
+ }
+ )
+
+ workflow = EntityCoverageWorkflow(
+ adapter=Mock(),
+ entity_labels=None,
+ entity_label_denylist=["email"],
+ )
+ result_df = workflow.postprocess(workflow.prepare(input_df))
+ missed = result_df[COL_MISSED_ENTITIES].iloc[0]
+ missed_labels = {e["label"] for e in missed}
+ assert "clinic_name" in missed_labels
+ assert "email" not in missed_labels
diff --git a/tests/test_measurement.py b/tests/test_measurement.py
index 5c9403df..7ea43832 100644
--- a/tests/test_measurement.py
+++ b/tests/test_measurement.py
@@ -446,6 +446,7 @@ def test_anonymizer_records_per_record_measurement_without_raw_pii(tmp_path: Pat
assert run_record["input_has_data_summary"] is False
assert run_record["detect"]["entity_label_source"] == "default"
assert run_record["detect"]["entity_label_count"] > 0
+ assert run_record["detect"]["entity_label_denylist"] is None
assert run_record["replace"]["strategy"] == "Redact"
assert run_record["replace"]["normalize_label"] is True
assert len(run_record["source_hash"]) == 64
@@ -456,6 +457,23 @@ def test_anonymizer_records_per_record_measurement_without_raw_pii(tmp_path: Pat
assert str(input_csv) not in serialized
+def test_detect_config_metadata_includes_entity_label_denylist() -> None:
+ from anonymizer.measurement.records.run import _detect_config_metadata
+
+ detect = Detect(entity_labels=["first_name", "email"], entity_label_denylist=["email"])
+ metadata = _detect_config_metadata(detect)
+ assert metadata["entity_label_denylist"] == ["email"]
+ assert metadata["entity_labels"] == ["email", "first_name"]
+
+
+def test_detect_config_metadata_denylist_none_when_not_set() -> None:
+ from anonymizer.measurement.records.run import _detect_config_metadata
+
+ detect = Detect()
+ metadata = _detect_config_metadata(detect)
+ assert metadata["entity_label_denylist"] is None
+
+
def test_anonymizer_measurement_config_writes_jsonl(tmp_path: Path) -> None:
input_csv = tmp_path / "input.csv"
output_jsonl = tmp_path / "measurements.jsonl"