Skip to content

docs: update notebooks for release #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lipikaramaswamy merged 2 commits into from
Jul 13, 2026
Merged

docs: update notebooks for release #218

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b6d3ea2
docs: update notebooks for release
lipikaramaswamy Jul 13, 2026
f243f14
docs: polish release notebooks
lipikaramaswamy Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/notebook_source/01_your_first_anonymization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# ---
# jupyter:
# jupytext:
Expand All @@ -15,6 +12,10 @@
# ---

# %% [markdown]
# <!--
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# -->
Comment thread
lipikaramaswamy marked this conversation as resolved.
# # 🕵️ Your First Anonymization
#
# Detect sensitive entities and replace them with LLM-generated substitutes --
Expand Down
31 changes: 19 additions & 12 deletions docs/notebook_source/02_inspecting_detected_entities.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# ---
# jupyter:
# jupytext:
Expand All @@ -15,6 +12,10 @@
# ---

# %% [markdown]
# <!--
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# -->
# # 🕵️ Inspecting Detected Entities
#
# Dig into the entity detection pipeline output -- what was detected,
Expand All @@ -26,6 +27,10 @@
# We use **Annotate** mode because it preserves the original text while tagging each entity
# with its label, making it ideal for reviewing detection quality.
#
# > **Privacy warning:** `Annotate` does not anonymize the text. Sensitive values
# > remain in the output, so use it only for inspection -- not as a privacy-safe
# > production strategy.
#
# #### 📚 What you'll learn
#
# - Run the detection pipeline and inspect its output using Annotate mode
Expand Down Expand Up @@ -100,13 +105,15 @@
# %% [markdown]
# ## 📋 Columns
#
# - `trace_dataframe` contains all internal columns from the pipeline
# (detection, validation, replacement, etc.).
# - `result.dataframe["final_entities"]` is the stable, public entity output.
# - `trace_dataframe` contains internal pipeline columns for deeper debugging;
# those underscore-prefixed columns may change between releases.

# %%
df = result.trace_dataframe
print(f"Records: {len(df)}")
print(f"Columns: {list(df.columns)}")
trace_df = result.trace_dataframe
final_entities = result.dataframe["final_entities"]
print(f"Records: {len(trace_df)}")
print(f"Columns: {list(trace_df.columns)}")

# %% [markdown]
# ## 🎯 Detected entities
Expand All @@ -116,7 +123,7 @@

# %%
row_idx = 0
raw = df.loc[row_idx, "_detected_entities"]
raw = final_entities.iloc[row_idx]
entities = raw["entities"] if isinstance(raw, dict) else raw
print(f"Record {row_idx}: {len(entities)} entities detected\n")

Expand All @@ -132,7 +139,7 @@

# %%
label_counts = Counter()
for raw in df["_detected_entities"]:
for raw in final_entities:
entity_list = raw["entities"] if isinstance(raw, dict) else raw
for entity in entity_list:
label_counts[entity["label"]] += 1
Expand All @@ -152,7 +159,7 @@

# %%
source_counts = Counter()
for raw in df["_detected_entities"]:
for raw in final_entities:
entity_list = raw["entities"] if isinstance(raw, dict) else raw
for entity in entity_list:
source_counts[entity.get("source", "unknown")] += 1
Expand All @@ -168,7 +175,7 @@

# %%
row_idx = 0
raw_bv = df.loc[row_idx, "_entities_by_value"]
raw_bv = trace_df.loc[row_idx, "_entities_by_value"]
by_value = raw_bv["entities_by_value"] if isinstance(raw_bv, dict) else raw_bv
print(f"Record {row_idx}: {len(by_value)} unique entity values\n")

Expand Down
8 changes: 5 additions & 3 deletions docs/notebook_source/03_choosing_a_replacement_strategy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# ---
# jupyter:
# jupytext:
Expand All @@ -15,6 +12,10 @@
# ---

# %% [markdown]
# <!--
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# -->
# # 🕵️ Choosing a Replacement Strategy
#
# Four [replace mode](../../concepts/replace/) strategies compared side-by-side on the same data.
Expand All @@ -25,6 +26,7 @@
# | **Redact** | Label-based markers (`[REDACTED_FIRST_NAME]`) |
# | **Annotate** | Tags entities but keeps original text |
# | **Hash** | Deterministic hash digest |
#
# #### 📚 What you'll learn
#
# - Compare **Redact**, **Annotate**, **Hash**, and **Substitute** on the same input
Expand Down
25 changes: 20 additions & 5 deletions docs/notebook_source/04_rewriting_biographies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# ---
# jupyter:
# jupytext:
Expand All @@ -15,16 +12,22 @@
# ---

# %% [markdown]
# <!--
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# -->
# # 🕵️ Rewriting Biographies
#
# Instead of replacing entities with tokens, rewrite mode generates a
# privacy-safe transformation of the entire text. The pipeline:
# privacy-safe transformation of the entire text. The `run()` / `preview()` pipeline:
Comment thread
binaryaaron marked this conversation as resolved.
#
# 1. Detects entities (same as replace mode, plus latent entity detection)
# 2. Classifies the domain and assigns sensitivity dispositions
# 3. Generates a rewritten version that obscures sensitive entities
# 4. Evaluates quality (utility) and privacy (leakage) with an automated repair loop
# 5. Runs a final optional LLM judge for informational scores
#
# Afterward, a separate optional `evaluate()` call runs LLM judges for
# detection validity and holistic privacy, quality, and style scores.
#
#
# #### 📚 What you'll learn
Expand Down Expand Up @@ -103,6 +106,8 @@
protect="All direct identifiers and quasi-identifier combinations (names, locations, employers, dates)",
preserve="Career trajectory, educational background, and professional accomplishments",
),
risk_tolerance="low",
max_repair_iterations=3,
),
)

Expand All @@ -125,6 +130,11 @@
preview.display_record(1)

# %% [markdown]
# > **How to interpret leakage:** Leakage is measured against the sensitivity
# > disposition. Details marked `leave_as_is` may remain without increasing
# > `leakage_mass`. If an output retains something you expected the privacy goal
# > to protect, inspect the Entity Disposition table.
#
# ## 🚀 Full run
#
# - `result.dataframe` has user-facing columns: rewritten text, scores, and the review flag.
Expand All @@ -145,6 +155,8 @@
# ## 🚩 Filter by review flag
#
# - Records where automated metrics exceed thresholds are flagged for manual review.

@binaryaaron binaryaaron Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent (review-pr): The threshold direction differs by metric

“Exceed” is accurate for leakage, but utility triggers review when it falls below its threshold. Could we say:

“Records that cross the configured leakage or utility thresholds are flagged for manual review.”

The same wording appears in notebook 05.

# - `needs_human_review` is threshold-based, so a record can have small nonzero
# leakage without being flagged.
# - Use this to prioritize human attention on the records that need it most.
# - See [Working with flagged records](../../concepts/rewrite/#working-with-flagged-records)
# for guidance on diagnosing and resolving flagged records.
Expand All @@ -159,6 +171,9 @@
# ## 🔬 Evaluate (optional)
#
# Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).
# Evaluation makes additional LLM calls per record. For larger datasets, evaluate
# a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.
# This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.

@binaryaaron binaryaaron Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent (review-pr): Clarify what may disagree

“Their” has no clear plural antecedent here. I suggest:

“The holistic privacy rubric and pipeline leakage metric are independent, so they may disagree.”

The same sentence appears in notebook 05.

# See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details.

# %%
Expand Down
21 changes: 18 additions & 3 deletions docs/notebook_source/05_rewriting_legal_documents.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# ---
# jupyter:
# jupytext:
Expand All @@ -15,6 +12,10 @@
# ---

# %% [markdown]
# <!--
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# -->
# # 🕵️ Rewriting Legal Documents
#
# Rewriting legal text (TAB dataset) with a domain-specific privacy goal
Expand Down Expand Up @@ -115,6 +116,8 @@
# ## 🎛️ Configure
#
# - `Detect(entity_labels=...)` overrides the default entity set with legal-specific labels.
# The explicit list is a strict allowlist for both detection and LLM augmentation:
# labels not included here are filtered out, so include every entity type you need.
# - `PrivacyGoal` tells the rewriter what to **protect** (identifiers, case numbers,
# institutional references) and what to **preserve** (legal reasoning, statutory references,
# ruling structure).
Expand Down Expand Up @@ -153,9 +156,16 @@
preview.display_record(1)

# %% [markdown]
# > **How to interpret leakage:** Leakage is measured against the sensitivity
# > disposition. Details marked `leave_as_is` may remain without increasing
# > `leakage_mass`. If an output retains something you expected the privacy goal
# > to protect, inspect the Entity Disposition table.
#
# ## 🚀 Full run
#
# - `result.dataframe` has user-facing columns: rewritten text, scores, and the review flag.
# - This notebook uses `risk_tolerance="minimal"`, which applies stricter repair
# and review thresholds than notebook 04.

# %%
result = anonymizer.run(config=config, data=input_data)
Expand All @@ -169,6 +179,8 @@
# ## 🚩 Filter by review flag
#
# - Records where automated metrics exceed thresholds are flagged for manual review.
# - The repair loop stops after `max_repair_iterations`; records that still need
# repair remain flagged for human review but are not pipeline failures.

@binaryaaron binaryaaron Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent (review-pr): Repair and human-review flags use different thresholds

This is not always true. _needs_repair uses repair_threshold, while needs_human_review is calculated separately from the final leakage, utility, and high-sensitivity-leak metrics. With risk_tolerance="minimal", for example, leakage mass 0.8 still needs repair because it exceeds 0.6, but it is not flagged solely for leakage because the review threshold is 1.0.

Suggested wording:

“The repair loop stops after max_repair_iterations. Afterward, needs_human_review is computed separately from the final leakage, utility, and high-sensitivity-leak metrics.”

# - Use this to prioritize human attention on the records that need it most.
# - See [Working with flagged records](../../concepts/rewrite/#working-with-flagged-records)
# for guidance on diagnosing and resolving flagged records.
Expand All @@ -183,6 +195,9 @@
# ## 🔬 Evaluate (optional)
#
# Call `evaluate()` to run LLM-as-judge scoring on the rewrite result — detection validity and three quality rubrics (privacy, quality, style).
# Evaluation makes additional LLM calls per record. For larger datasets, evaluate
# a preview first; this tutorial evaluates all 25 rows to demonstrate the complete workflow.
# This holistic judge is independent of pipeline leakage scoring, so their assessments may differ.
# See [Evaluation](../../concepts/evaluation/#rewrite-evaluation) for details.

# %%
Expand Down
Loading
Loading