feat(rag): govern perturbation anchors

[seonghobae]

Buyer-visible RAG validation gap

Issue #607 requires deterministic construct-specific perturbation anchors so reference-free RAG validation is not reduced to raw judge averages or ungoverned prompt experiments. Protected main already has the governed RAG request/provenance boundary; this Draft implements the next bounded, provider-free validation-contract slice.

Implemented contract

fast_mlsirm.scoring.rag provides an immutable, source-free, content-addressed RAGPerturbationAnchor contract with finite preregistered perturbation semantics:

• unsupported claim -> grounded_generation decreases;
• explicit contradiction -> grounded_generation decreases;
• irrelevant context -> retrieval_relevance decreases;
• required-evidence removal -> coverage_or_completeness_proxy decreases;
• citation-target swap -> citation_attribution decreases;
• semantic query paraphrase -> robustness invariant;
• style-only rewrite -> robustness invariant; and
• unanswerable query -> answerability_and_abstention increases.

Baseline and perturbed governed request fingerprints are mandatory, normalized and distinct. Unknown perturbations, malformed fingerprints and no-op request pairs fail closed. The canonical anchor fingerprint covers immutable anchor content through the shared contract machinery. Raw query/context/answer/source text remains absent from this contract surface.

Scientific / ownership boundary

This change is validation/provenance marshalling only. It introduces no provider call, metric arithmetic, thresholding, retrieval, calibration numerical kernel, truth adjudication or raw-content persistence. Expected direction is a preregistered validation hypothesis, not evidence that an observed system moved as expected. Human/AI/LLM/rule evaluator outputs remain fallible observations; numerical/psychometric production ownership remains Rust-first.

Exact-current stack evidence

• base branch: codex/fast-judge-accepted-type-regression at 11291fd5f03cf24e403289b0466a7b633d0d3971;
• exact Draft head: 02702f79cadc4f9926960a2d8393a40b90284d89;
• branch was non-destructively refreshed onto that exact predecessor head with no force push;
• exact-head CI run 31548664197: success, including Python 3.12/3.14, Rust/PyO3, package/reinstall/release-acceptance, explicit GPU smoke and fuzz;
• the old OpenCode implementation handoff was bound to predecessor head 4c2838f78bb806af372b7d57404285800f0448f5 and is stale after the current head/base movement; it is not an active current-head writer lease.

Earlier checkout/setup failures and 403 rerun attempts are predecessor infrastructure evidence only and are not current defects or current passing evidence.

Merge boundary

Keep Draft while #778 remains the predecessor. After #778 integrates, reconcile this unique RAG slice onto resulting protected main, regenerate every base-sensitive CI/security/review gate on the unchanged final head, resolve any valid current finding, and merge only under actual live repository policy. This contract alone does not establish RAG system validity, a scalar quality score, evaluator interchangeability, retrieval recall, world correctness, calibration, DIF/invariance or release readiness.

Advances #607.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c8b75ea6-c2d5-48c1-a1fc-906391337814

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@opencode-agent address

Bounded GREEN implementation handoff for Draft #777. Immediately before any source write, refetch exact source head 4c2838f78bb806af372b7d57404285800f0448f5, protected main c91ae210bc8e0340a5b0587cf5289b82a2e8ab92, reviews/comments and writer state; abort this lane if either ref moved incompatibly or another writer is active on python/fast_mlsirm/scoring/rag.py / the new test.

First run tests/test_scoring_rag_perturbation_anchors.py and preserve the realistic import/public-contract RED: the new finite perturbation/direction types and builder do not yet exist. Implement the narrowest GREEN under the existing fast_mlsirm.scoring.rag authority:

1. Add finite RAGPerturbationKind and RAGPerturbationDirection string enums exactly matching the test contract.
2. Add one immutable, content-addressed, source-free perturbation-anchor contract plus build_rag_perturbation_anchor(...). Reuse existing CanonicalContract, artifact_digest, descriptive_identifier, fingerprint, enum_value and stable assessment_error patterns rather than inventing another serialization hierarchy.
3. The builder accepts only anchor_id, baseline/perturbed governed request fingerprints and perturbation kind. Map each kind deterministically to the preregistered construct/direction in the RED. Reject identical request fingerprints with code identical_rag_perturbation_requests, unknown kinds through the normal invalid_rag_perturbation_kind enum error, and malformed fingerprints through field-specific shared validation. Do not permit caller override of expected construct/direction.
4. Expose deterministic anchor_fingerprint and to_dict() with normalized values. Keep raw query/context/answer/source text and provider output entirely absent. This is validation provenance only: no score comparison, threshold, metric, retrieval, inference, truth adjudication, calibration arithmetic or claim that a run passed its expected direction.
5. Preserve existing RAG request behavior/tests. Add complete public docstrings and any direct immutability/replay tests needed for 100% added contract statement/branch coverage.
6. After GREEN, add concise APA-linked doctoring/changelog for the controlled perturbation validation role, render/check managed CHANGELOG.md, run focused RAG/scoring tests + Ruff/format and then complete applicable Python/package/security gates on one unchanged head. Do not widen into providers, RAGAS dependency, new likelihood, architecture rewrite, version/release or hosted persistence.

Keep Draft until exact-head evidence is complete; then stop writing for independent review/merge.

[seonghobae]

Superseded by surgical GREEN on current main.