feat(rag): govern perturbation anchors for privacy-preserving scoring

[seonghobae]

Surgical re-apply of draft #777 unique RAG paths onto current main. Local 13 tests green. Merge when product gates pass.

Summary by CodeRabbit

• New Features

  • Added governed RAG perturbation anchors with deterministic, content-addressed identities.
  • Supports multiple perturbation types with defined expected constructs and validation directions.
  • Added safeguards for malformed fingerprints, unsupported semantics, and identical baseline and perturbed requests.
  • Anchors can be serialized for reliable validation and comparison.

• Documentation

  • Expanded guidance on privacy, provenance, research standards, and the scope and limitations of RAG scoring anchors.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 35a1d92c-ae6d-4e92-b04b-9cac60beec6b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The PR adds governed RAG perturbation anchors. It defines finite perturbation semantics, validates immutable content-addressed identities, exposes a builder API, documents privacy boundaries, and adds fail-closed contract tests.

Changes

RAG perturbation anchors

Layer / File(s)	Summary
Perturbation vocabulary and contract docs/doctoring/rag_scoring_request_privacy.md, docs/changelog.d/777-rag-perturbation-anchors.md, python/fast_mlsirm/scoring/rag.py	The documentation and enums define supported perturbation kinds, expected directions, fingerprint requirements, privacy boundaries, and fail-closed behavior.
Immutable anchor construction python/fast_mlsirm/scoring/rag.py	RAGPerturbationAnchor validates request fingerprints, derives expected semantics, serializes canonical identity data, computes deterministic fingerprints, and is exposed with its builder.
Anchor contract validation tests/test_scoring_rag_perturbation_anchors.py	Tests cover supported values, deterministic identity, serialization, distinct requests, malformed inputs, unknown semantics, and exclusion of raw RAG content from the builder API.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant AnchorBuilder
  participant RAGPerturbationAnchor
  participant ArtifactDigest
  Caller->>AnchorBuilder: provide anchor ID and request fingerprints
  AnchorBuilder->>RAGPerturbationAnchor: validate and construct anchor
  RAGPerturbationAnchor->>ArtifactDigest: fingerprint canonical anchor content
  ArtifactDigest-->>RAGPerturbationAnchor: return deterministic fingerprint
  RAGPerturbationAnchor-->>Caller: return immutable anchor

Loading

Possibly related PRs

• ContextualWisdomLab/fast-mlsirm#533: Shares deterministic, immutable, content-addressed fingerprints and fail-closed validation.
• ContextualWisdomLab/fast-mlsirm#613: Directly modifies python/fast_mlsirm/scoring/rag.py and shares the governed fingerprint-based RAG request contract.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: governed RAG perturbation anchors for privacy-preserving scoring.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/rag-perturbation-anchors-green

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@cwl-noema-review

Independent exact-head review requested for 8e542e084fa56ae355a7cdb070b021cd8721f931 against current protected main 5231438df8e02bd71d88e692aaaa09a1d6c8cdc3.

Review only the four-path privacy-preserving RAG perturbation-anchor contract. Verify immutable/content-addressed anchor provenance, strict source/evidence binding, bounded caller-controlled collections, deterministic ordering, privacy/non-retention claims, and that perturbation anchors remain governed evidence rather than a scoring truth or validity claim. Exact-head CI, Security Scan, SAST and CodeRabbit status are successful and no review threads exist. Read-only review; do not modify or merge.

[seonghobae]

@opencode-agent

Perform a read-only exact-head review of 8e542e084fa56ae355a7cdb070b021cd8721f931 against protected main 5231438df8e02bd71d88e692aaaa09a1d6c8cdc3. Exact-head CI, Security Scan, SAST and CodeRabbit status are successful and no review threads exist. Review only the privacy-preserving RAG perturbation-anchor contract for provenance, source/evidence binding, bounded collections, deterministic behavior and conservative claims. Do not modify or merge.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/doctoring/rag_scoring_request_privacy.md`:
- Around line 5-13: Update the references section in
rag_scoring_request_privacy.md by adding a concise mapping section that links
each perturbation direction to primary-source support, covering citation
attribution, evidence removal, robustness invariance, and abstention in addition
to the RAGAS/ARES-supported faithfulness, context relevance, and answer
relevance. If primary-source support cannot be established for any mapping, move
that mapping to an explicit model-design PR instead of presenting it as
literature-backed.

In `@python/fast_mlsirm/scoring/rag.py`:
- Around line 194-212: Update build_rag_perturbation_anchor to resolve the
canonical governed requests identified by baseline_request_fingerprint and
perturbed_request_fingerprint, then validate their kind-specific provenance and
evidence relationship before constructing RAGPerturbationAnchor or deriving its
identity. Reject unrelated request pairs, and add negative tests covering
invalid relationships for each applicable perturbation kind.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9fe00c30-8e17-4f39-a88e-3b3eaa9e74fb

📥 Commits

Reviewing files that changed from the base of the PR and between 1d4bad3 and 8317fe8.

📒 Files selected for processing (4)

• docs/changelog.d/777-rag-perturbation-anchors.md
• docs/doctoring/rag_scoring_request_privacy.md
• python/fast_mlsirm/scoring/rag.py
• tests/test_scoring_rag_perturbation_anchors.py