Skip to content

feat(enterprise): parse explicit source values - #533

Merged
seonghobae merged 103 commits into
mainfrom
feat/enterprise-explicit-value-parser
Aug 5, 2026
Merged

feat(enterprise): parse explicit source values#533
seonghobae merged 103 commits into
mainfrom
feat/enterprise-explicit-value-parser

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Advances issue #404 with the next reviewable enterprise-evidence vertical slice after #529 and #532.

  • adds a deterministic, provider-neutral parser for explicit Gregorian dates, marked deadlines, allowlisted currency amounts, bounded recurrence frequencies, and labeled customer/account identifiers;
  • verifies transient source text against the exact EnterpriseSourceRecord content fingerprint and Python Unicode-code-point character count before parsing;
  • hashes customer/account identifiers before public record construction and retains neither raw source text nor clear-text identifiers;
  • emits immutable, content-addressed ExplicitValueRecord values and compiles exact occurrences into the accepted EvidenceSpanRecord boundary without creating competing scoring, observation, result, or engine schemas;
  • enforces deterministic ordering, deadline precedence, overlap rejection, record limits during candidate collection, strict payload and metadata validation, exact span fingerprints, and fail-closed custom-provider output reconstruction;
  • rejects provider-owned subclasses, mutated records, source-revision mismatches, out-of-bounds spans, duplicate/overlapping output, malformed payloads, and untrusted callback failures without leaking provider details;
  • preserves conservative interpretation: extraction proves only that matching text occurred in one verified source revision, not truth, currency, materiality, probability, priority, construct validity, or causal effect.

Refs #404

Validation

  • 73 focused parser and provider-boundary tests pass on the hardened implementation;
  • focused explicit_values.py evidence is 100% statements and 100% branches (294 statements, 122 branches, no missing lines or partial branches);
  • tests cover deterministic normalization, Unicode offsets, exact source replay, privacy, identifier hashing, overlap and limit behavior, malformed/adversarial payloads, custom-parser canonical reconstruction, subclass and mutation rejection, exception redaction, protocol substitution, defensive payload short-circuit branches, and changelog parity;
  • Ruff lint and formatting, Python compilation, and authoritative changelog rendering/checks pass in the verified review-fix run;
  • public contracts and parser surfaces include complete docstrings;
  • authoritative documentation and CHANGELOG.md include the parser boundary, interpretation limits, whole-document fail-closed scope, and APA 7th references to ISO 8601-1, ISO 4217, ISO/IEC 42001, NIST AI RMF, Python standard-library semantics, and Unicode text segmentation;
  • release version is unchanged because this is one issue Implement decision-conditional enterprise issue intelligence vertical #404 slice and not yet a standalone release cut.

Exact-head CI, Security Scan, SAST, repository coverage, packaging, and final review remain merge gates.

Summary by CodeRabbit

  • New Features

    • Added deterministic parsing of explicitly stated dates, deadlines, currency amounts, recurrence frequencies, and customer identifiers.
    • Added privacy-preserving identifier handling, evidence spans, normalized values, and bounded output.
    • Added validation for source integrity, overlaps, ordering, custom parser results, and malformed input.
  • Documentation

    • Documented supported formats, privacy and security behavior, standards coverage, and parser contracts.
    • Updated the changelog and historical release references.
  • Tests

    • Added comprehensive coverage for parsing, validation, Unicode handling, limits, redaction, and provider safety.

Copy link
Copy Markdown
Contributor Author

@jules Implement the full draft slice on this branch from docs/superpowers/plans/2026-08-05-enterprise-explicit-value-parser.md. Preserve all accepted #529/#532 contracts and package exports. Add ExplicitValueKind, immutable ExplicitValueRecord, runtime-checkable EnterpriseExplicitValueParser, DeterministicExplicitValueParser, and parse_enterprise_explicit_values() under fast_mlsirm.scoring.enterprise_issue; compile exact matches to the existing EvidenceSpanRecord boundary; retain no raw source text or clear-text customer/account identifier; verify source SHA-256 and Python Unicode-code-point character count; use deterministic non-overlapping extraction for ISO YYYY-MM-DD, marked deadlines, caller-allowlisted ISO 4217 alphabetic currency codes with string-to-Decimal normalization, explicit frequency forms, and labeled customer/account identifiers hashed before persistence. Deadline must supersede its embedded calendar-date match. Fail closed on impossible date-shaped values, malformed configuration, source replay/count mismatch, unresolved overlaps, and record-limit overflow. Add complete public docstrings, property/metamorphic/security tests, exact offset/fingerprint tests, protocol tests, evidence-role delegation tests, 100% statement and branch coverage for added code, package exports, buyer-facing documentation, APA 7th references, an authoritative changelog fragment, and rendered CHANGELOG.md. Run Ruff format/lint, typing/import checks, focused tests, coverage with --branch --fail-under=100, docstring checks, changelog parity, and the feasible repository test suite. Do not add semantic extraction, sentiment, scoring, calibration, ranking, utility, causal, or queue arithmetic. Commit objective fixes directly and report exact commands/results; keep the PR draft until all validations are green.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ba49e271-c4e5-4a8d-99b1-780f3b6190d2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a deterministic, provider-neutral parser for explicit enterprise values. It creates immutable, content-addressed records for dates, deadlines, currencies, frequencies, and customer identifiers. It validates spans, ordering, limits, metadata, and custom-provider output.

Changes

Enterprise explicit-value parsing

Layer / File(s) Summary
Contracts and canonical records
docs/enterprise_issue_evidence_contracts.md, docs/superpowers/plans/..., python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py, python/fast_mlsirm/scoring/enterprise_issue/__init__.py
Defines public parser types, immutable records, supported value kinds, validation rules, fingerprints, evidence conversion, and package exports.
Deterministic extraction and evidence compilation
python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py, tests/test_scoring_enterprise_explicit_values.py, docs/changelog.d/enterprise-explicit-value-parser.md, CHANGELOG.md
Extracts supported values from verified source text, normalizes payloads, hashes identifiers, rejects overlaps, orders results deterministically, and enforces limits.
Provider validation and fail-closed entry point
python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py, tests/test_scoring_enterprise_explicit_values.py, tests/test_scoring_enterprise_explicit_values_provider_boundary.py
Revalidates custom-provider records against the source, rejects malformed or duplicated output, bounds candidate collection, reconstructs records, and redacts provider errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SourceRecord
  participant Parser
  participant CustomProvider
  participant ExplicitValueRecord
  participant EvidenceSpanRecord
  SourceRecord->>Parser: verify source replay and span offsets
  Parser->>CustomProvider: request explicit-value records
  CustomProvider-->>Parser: return candidate records
  Parser->>ExplicitValueRecord: reconstruct and validate canonical records
  ExplicitValueRecord->>EvidenceSpanRecord: convert accepted records to evidence spans
Loading

Suggested reviewers: claude

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding enterprise parsing for explicit source values.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/enterprise-explicit-value-parser

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@jules Exact-head CI on 1cb60090f393c06e2b7201a3f1a9aa0a00e0c6c7 found two objective failures. Please commit both fixes directly to this branch, then run focused tests before the full suite:

  1. python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py::_decimal_amount() raises AttributeError for a non-string object before its structured error boundary. Reject non-str values (or include AttributeError without reflecting input) so test_private_normalizers_cover_nonpublic_exception_boundaries receives AssessmentSpecError: invalid_decimal_amount.
  2. Render the authoritative fragment into CHANGELOG.md with python scripts/render_changelog_fragments.py --update CHANGELOG.md; test_repository_changelog_is_rendered_from_current_fragments is failing because the new fragment is not yet represented.

Then run at minimum:
pytest tests/test_scoring_enterprise_explicit_values.py tests/test_changelog_fragment_contract.py
python scripts/render_changelog_fragments.py --check CHANGELOG.md
Report exact results and keep the PR draft until exact-head CI is green.

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review Please fix the two exact-head CI failures on this branch without weakening tests: (1) make _decimal_amount() reject non-string input through structured invalid_decimal_amount rather than leaking AttributeError; (2) run python scripts/render_changelog_fragments.py --update CHANGELOG.md and commit the rendered file. Verify the two focused test files and changelog --check, then report exact results.

Comment thread .github/workflows/pr-533-autofix.yml Fixed

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Comment thread .github/workflows/pr-533-payload-hardening.yml Fixed

Copy link
Copy Markdown
Contributor Author

@cwl-noema-review Review the exact current implementation after the payload-hardening workflow completes. Focus on fail-closed construction for non-mapping or adversarial payloads, provider-protocol output validation against the exact source revision/span, metadata privacy, deterministic ordering/overlap rules, complete branch coverage, and conservative statistical/product claims. Commit only valid fixes directly to this branch, remove any one-shot workflow from the final tree, and report exact validation results.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@seonghobae
seonghobae enabled auto-merge (squash) August 5, 2026 01:44

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review Final exact-head review requested for 7ec03f5d6b53bffe162d09036930c9408240f89f. Review only the eight current product/docs/test files; all one-shot workflows and helper scripts are absent. Validate provider-boundary reconstruction, source/span replay, identifier privacy, bounded deterministic collection, structured error redaction, 100% statement/branch evidence, complete docstrings, APA 7 documentation, and changelog parity. Do not approve or merge unless this exact head's CI, Security Scan, SAST, unresolved-thread, and repository-policy gates pass.

Comment thread .github/workflows/pr-533-restore-changelog.yml Fixed

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review Final exact-head review requested for 1e194b3df5e4686f1b2ea5baae201c559773b866. Review only the eight current product/docs/test files. Verify the deterministic provider-neutral parser boundary, exact source/span replay, privacy-preserving identifier hashing, bounded and canonical provider reconstruction, structured error redaction, 100% statement/branch and docstring evidence, preserved release history, APA 7 traceability, and changelog fragment parity. Do not approve or merge unless this exact head's CI, Security Scan, SAST, unresolved-thread, and repository-policy gates pass.

@seonghobae
seonghobae merged commit 3d0d27c into main Aug 5, 2026
32 checks passed
@seonghobae
seonghobae deleted the feat/enterprise-explicit-value-parser branch August 5, 2026 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants