Repository navigation
feat(enterprise): parse explicit source values #533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
103 commits
Select commit
Hold shift + click to select a range
967a3af
docs(plan): specify enterprise explicit-value parser
seonghobae 851837e
feat(enterprise): add deterministic explicit-value parser
seonghobae f5148d8
feat(enterprise): export explicit-value parser contracts
seonghobae 1d09b67
test(enterprise): verify explicit-value parsing and privacy
seonghobae 8bb61a8
docs(enterprise): document explicit-value parser boundaries
seonghobae 1cb6009
docs(changelog): add enterprise parser release note
seonghobae 322da69
ci: add one-shot PR 533 autofix
seonghobae 7ab32a2
fix(enterprise): close parser validation gaps
github-actions[bot] 24fad2e
docs(enterprise): document structured decimal validation
seonghobae 4046e6b
ci: retrigger exact-head validation
seonghobae 00ddba8
ci: validate repaired exact head
seonghobae 606a0da
ci: harden PR 533 payload validation
seonghobae ad3cdd6
ci: repair payload hardening workflow
seonghobae ffb8459
ci: simplify payload hardening workflow
seonghobae 2058411
fix(enterprise): harden explicit payload validation
github-actions[bot] c2a5917
ci: retrigger hardened exact head
seonghobae 5fe53a1
ci: validate hardened exact head
seonghobae fbd4b02
docs(enterprise): record fail-closed payload validation
seonghobae ced912d
ci: validate PR 533 provider output
seonghobae a9d82b7
fix(enterprise): validate explicit parser output
github-actions[bot] 95e21ae
ci: add one-shot provider boundary hardening
seonghobae 857980e
ci: trigger provider boundary hardening
seonghobae bfbbfd3
ci: repair PR 533 provider hardening
seonghobae 28d85c3
fix(enterprise): validate provider parser output
github-actions[bot] 040d7a7
ci: validate provider-hardened exact head
seonghobae 11e656e
ci: repair PR 533 metadata indentation
seonghobae 17c385d
fix(enterprise): restore metadata validation indentation
github-actions[bot] 856daf6
ci: stage PR 533 indentation repair
seonghobae 6dd2ad6
ci: trigger metadata indentation repair
seonghobae a8c08a7
docs(enterprise): specify provider output validation
seonghobae 5850eef
ci: remove completed indentation repair workflow
seonghobae 25845ff
ci: validate repaired provider boundary
seonghobae 0ced3f7
docs(enterprise): enumerate provider validation tests
seonghobae 6deb609
ci: stage provider exception hardening
seonghobae 40e23af
ci: trigger provider exception hardening
seonghobae 36e6541
ci: trigger provider exception hardening
seonghobae 1c8b626
ci: repair provider exception hardening
seonghobae f50584a
ci: simplify provider exception hardening
seonghobae d3218b6
ci: trigger provider exception hardening
seonghobae 2a8aa19
ci: trigger provider boundary hardening
seonghobae b62305b
ci: trigger provider boundary hardening
seonghobae d9911b3
chore: trigger provider boundary hardening
seonghobae e319f57
ci: trigger provider hardening on ready review
seonghobae e1318f5
ci: add one-shot provider hardening helper
seonghobae 4ef6670
ci: prepare provider hardening runtime
seonghobae 422bd39
ci: run provider hardening on PR synchronization
seonghobae 4edc689
ci: fix exact-head provider hardening helper
seonghobae 36ba330
ci: prepare provider hardening runtime
seonghobae 8ae45a6
ci: validate provider hardening with locked test dependencies
seonghobae c5b00b6
ci: repair provider hardening callback generation
seonghobae 5713d19
fix(enterprise): seal provider trust boundary
github-actions[bot] ed2184f
ci: retrigger provider-hardened validation
seonghobae 56f29e4
docs: record provider boundary hardening
seonghobae 07a0455
ci: validate provider-hardened exact head
seonghobae 4cba5dc
docs: synchronize provider hardening changelog
seonghobae 0e6b935
ci: apply verified explicit-value review fixes
seonghobae 3e0243b
ci: repair explicit-value review fixture
seonghobae 943e92d
fix(review): address CodeRabbit parser findings
seonghobae ffc4114
fix(review): address CodeRabbit parser findings
seonghobae 5683236
ci: remove failed review repair workflow
seonghobae 4a799fb
ci: remove superseded review hardening workflow
seonghobae ba69101
ci: remove superseded review fixes workflow
seonghobae ffbc1ca
ci: remove stale review-fix trigger
seonghobae 553f7db
chore: stage deterministic review fix script
seonghobae f27e640
ci: run deterministic PR 533 review fixes
seonghobae 0e74378
ci: run deterministic PR 533 review fixes
seonghobae 598b28c
ci: repair PR 533 review workflow syntax
seonghobae c043d9b
ci: run deterministic PR 533 review fixes
seonghobae 49890e2
ci: run accepted PR 533 review fixes
seonghobae b2354b7
ci: format accepted PR 533 review fixes before validation
seonghobae 165c63e
ci: retrigger accepted PR 533 review fixes
seonghobae 557c2fd
ci: complete verified PR 533 review remediation
seonghobae b08ac01
ci: install PR 533 focused coverage tooling
seonghobae f096349
ci: run focused coverage without package import aliasing
seonghobae cc10f9b
ci: isolate PR 533 review validation
seonghobae 3403f41
test: cover explicit-value payload guard branches
seonghobae 83eca8f
test: stage PR 533 branch-coverage fixes
seonghobae 8625b95
ci: repair targeted payload coverage workflow syntax
seonghobae a770192
ci: cover defensive PR 533 review branches
seonghobae 3a738f7
fix(enterprise): address explicit-value review findings
github-actions[bot] febf376
ci: remove temporary PR 533 repair workflow
seonghobae fc44b59
chore: remove PR 533 repair trigger
seonghobae 5677dc7
chore: remove temporary PR 533 coverage helper
seonghobae 71da90f
chore: remove temporary PR 533 review helper
seonghobae b8183dc
test: cover remaining explicit-value payload branches
seonghobae 6ab8cfd
ci: apply final explicit-value branch tests
seonghobae e05fb08
ci: avoid blank line at test file EOF
seonghobae b038035
test(enterprise): cover remaining explicit-value branches
github-actions[bot] 7ec03f5
chore(ci): remove completed PR 533 review fixer
seonghobae 01f33a6
ci: restore immutable changelog history for PR 533
seonghobae 37b5ebe
fix(changelog): restore immutable release history
github-actions[bot] afd9ace
chore(ci): remove completed changelog restorer
seonghobae 872e2ee
ci: restore PR 533 changelog history
seonghobae 8766b88
docs: restore complete changelog history
github-actions[bot] c084865
docs: clarify explicit-value offset boundary
seonghobae eaa6011
ci: synchronize PR 533 changelog
seonghobae 1963676
docs: synchronize explicit-value changelog fragment
github-actions[bot] 3ae2bcc
ci: render PR 533 changelog parity
seonghobae 0a72280
docs: require changelog synchronization after fragment changes
seonghobae efb459d
docs(changelog): render explicit-value parser fragment
github-actions[bot] 8d62d69
docs: mark changelog parity as merge gate
seonghobae acb3569
docs(plan): record exact-head verification gate
seonghobae 1e194b3
chore(ci): refresh exact-head protected checks
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| name: PR 533 one-shot autofix | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - feat/enterprise-explicit-value-parser | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| concurrency: | ||
| group: pr-533-one-shot-autofix | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| fix: | ||
| if: github.repository == 'ContextualWisdomLab/fast-mlsirm' | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 20 | ||
| steps: | ||
| - name: Check out exact branch head | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 | ||
| with: | ||
| ref: feat/enterprise-explicit-value-parser | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Configure Python | ||
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Apply structured validation and render changelog | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| python - <<'PY' | ||
| from pathlib import Path | ||
|
|
||
| path = Path("python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py") | ||
| text = path.read_text(encoding="utf-8") | ||
| before = '''def _decimal_amount(value: str, path: str) -> str: | ||
| """Return exact finite nonnegative decimal text without float conversion.""" | ||
| try: | ||
| amount = Decimal(value.replace(",", "")) | ||
| ''' | ||
| after = '''def _decimal_amount(value: str, path: str) -> str: | ||
| """Return exact finite nonnegative decimal text without float conversion.""" | ||
| if type(value) is not str: | ||
| raise assessment_error( | ||
| "invalid_decimal_amount", | ||
| path, | ||
| "decimal amount must use the accepted nonnegative decimal grammar", | ||
| ) | ||
| try: | ||
| amount = Decimal(value.replace(",", "")) | ||
| ''' | ||
| if before not in text: | ||
| raise SystemExit("expected _decimal_amount implementation was not found") | ||
| path.write_text(text.replace(before, after, 1), encoding="utf-8") | ||
| PY | ||
| python scripts/render_changelog_fragments.py --update CHANGELOG.md | ||
| python scripts/render_changelog_fragments.py --check CHANGELOG.md | ||
|
|
||
| - name: Install focused test dependencies | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| python -m pip install --require-hashes -r requirements/ci.txt | ||
| python -m pip install --no-deps --no-build-isolation -e . | ||
|
|
||
| - name: Verify focused regressions | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| pytest tests/test_scoring_enterprise_explicit_values.py tests/test_changelog_fragment_contract.py | ||
|
|
||
| - name: Commit fixes and remove one-shot workflow | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "41898282+github-actions[bot]@users.noreply.github.com" | ||
| git rm .github/workflows/pr-533-autofix.yml | ||
| git add CHANGELOG.md python/fast_mlsirm/scoring/enterprise_issue/explicit_values.py | ||
| git diff --cached --check | ||
| git commit -m "fix(enterprise): close parser validation gaps" | ||
| git push origin HEAD:feat/enterprise-explicit-value-parser | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Deterministic enterprise explicit-value parser | ||
|
|
||
| ## Added | ||
|
|
||
| - Added a provider-neutral deterministic parser for verified explicit calendar | ||
| dates, marked deadlines, allowlisted currency amounts, recurrence frequencies, | ||
| and labeled customer or account identifiers under | ||
| `fast_mlsirm.scoring.enterprise_issue`. | ||
| - Added exact source fingerprint and Python Unicode-code-point-count replay, | ||
| UTF-8 span fingerprints, deterministic deadline precedence, overlap rejection, | ||
| bounded output, and hashed identifier payloads without retaining source text or | ||
| clear-text customer identifiers. | ||
| - Added immutable content-addressed records that compile exact occurrences into | ||
| the existing directly stated `EvidenceSpanRecord` boundary without adding a | ||
| parallel scoring, observation, result, or engine schema. | ||
| - Added deterministic, protocol, privacy, security, exact-decimal, offset, | ||
| ordering-invariance, metamorphic sentiment-independence, and fail-closed tests, | ||
| plus APA 7th standards traceability and conservative interpretation limits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
136 changes: 136 additions & 0 deletions
136
docs/superpowers/plans/2026-08-05-enterprise-explicit-value-parser.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,136 @@ | ||
| # Enterprise explicit-value parser implementation plan | ||
|
|
||
| ## Goal | ||
|
|
||
| Advance issue #404 with the smallest reviewable parser slice after the governed | ||
| enterprise evidence and scoring-request contracts. Add a deterministic, | ||
| provider-neutral boundary for explicit values already present in authorized source | ||
| text. Do not perform semantic issue extraction, sentiment analysis, scoring, | ||
| calibration, ranking, utility arithmetic, causal inference, or queue routing. | ||
|
|
||
| ## Accepted boundaries | ||
|
|
||
| The implementation belongs under `fast_mlsirm.scoring.enterprise_issue` and must | ||
| reuse `EnterpriseSourceRecord`, `EvidenceSpanRecord`, | ||
| `EnterpriseAssertionKind`, and the shared scoring contracts. It must not create a | ||
| parallel observation, result, engine, assessment, or rubric schema. | ||
|
|
||
| The parser receives source text transiently, verifies it against the exact | ||
| `EnterpriseSourceRecord` content fingerprint and character count, and returns | ||
| content-addressed records without retaining source text or clear-text customer | ||
| identifiers. | ||
|
|
||
| ## Public contract | ||
|
|
||
| Add: | ||
|
|
||
| - `ExplicitValueKind`, with distinct values for `calendar_date`, | ||
| `deadline_date`, `money_amount`, `frequency_count`, and | ||
| `customer_identifier`; | ||
| - immutable `ExplicitValueRecord`, containing the exact source-record | ||
| fingerprint, UTF-8 source-span fingerprint, Python Unicode-code-point offsets, | ||
| kind-specific normalized payload, and parser-revision fingerprint; | ||
| - runtime-checkable `EnterpriseExplicitValueParser` protocol; | ||
| - `DeterministicExplicitValueParser`, whose parser revision is derived from its | ||
| patterns and configured ISO 4217 alphabetic currency-code allowlist; | ||
| - `parse_enterprise_explicit_values()` as the stable convenience boundary. | ||
|
|
||
| `ExplicitValueRecord.to_evidence_span()` must compile the exact span into the | ||
| accepted `EvidenceSpanRecord` boundary as a directly stated fact. This means only | ||
| that the value was explicitly present in the source, not that its real-world | ||
| content is true. | ||
|
|
||
| ## Deterministic grammar | ||
|
|
||
| The first implementation recognizes only deliberately narrow, auditable forms: | ||
|
|
||
| 1. Gregorian calendar dates in extended `YYYY-MM-DD` form, validated by | ||
| `date.fromisoformat()`; | ||
| 2. deadlines expressed by an explicit marker such as `due`, `deadline`, `by`, | ||
| or `no later than`, followed by an accepted calendar date; | ||
| 3. money amounts expressed as an allowlisted three-letter ISO 4217 alphabetic | ||
| code followed by an exact decimal amount; commas may be grouping separators, | ||
| and values are normalized through `Decimal` constructed from text; | ||
| 4. frequencies such as `3 times per month`, `4 incidents per week`, or | ||
| `2 occurrences/year`, normalized to an integer count and one of `day`, | ||
| `week`, `month`, `quarter`, or `year`; | ||
| 5. customer/account identifiers introduced by an explicit `customer_id`, | ||
| `customer id`, `account_id`, or `account id` label. The normalized payload | ||
| stores only SHA-256 over the identifier token, never the token itself. | ||
|
|
||
| Deadline matches take precedence over the embedded calendar-date match. Other | ||
| overlapping matches are rejected or deterministically resolved by documented | ||
| kind priority; the same source occurrence must never be multiplied silently. | ||
| Results are sorted by `(start_offset, end_offset, value_kind, | ||
| explicit_value_fingerprint)` and are independent of pattern declaration order. | ||
|
|
||
| ## Fail-closed validation | ||
|
|
||
| Reject: | ||
|
|
||
| - non-`EnterpriseSourceRecord` inputs; | ||
| - non-string source text; | ||
| - source-content fingerprint or character-count mismatch; | ||
| - syntactically date-shaped but impossible Gregorian dates; | ||
| - malformed configured currency codes; | ||
| - non-finite or signed money amounts outside the accepted grammar; | ||
| - non-positive frequency counts; | ||
| - empty or oversized customer identifiers; | ||
| - overlapping accepted records that cannot be resolved by the declared deadline | ||
| precedence; | ||
| - record counts above the bounded parser limit; | ||
| - caller metadata containing raw/sensitive source fields. | ||
|
|
||
| All public identifiers must remain descriptive, nonnumeric, and at least two | ||
| snake-case tokens. Public docstrings and statement/branch coverage for added code | ||
| must be complete. | ||
|
|
||
| ## Tests | ||
|
|
||
| Add deterministic tests for: | ||
|
|
||
| - exact extraction of all five kinds from one mixed Unicode source; | ||
| - source slices matching returned offsets while serialized records retain no raw | ||
| source text or customer token; | ||
| - deadline/date overlap producing one deadline record; | ||
| - parser output and fingerprints remaining invariant to configured currency-code | ||
| input order; | ||
| - exact decimal normalization without binary-float conversion; | ||
| - invalid date, source replay, source length, protocol, payload, overlap, and | ||
| maximum-record failures; | ||
| - conversion to the accepted `EvidenceSpanRecord` and shared | ||
| `EvidenceReference` role; | ||
| - no sentiment wording effect on extracted explicit values when the explicit | ||
| evidence is unchanged; | ||
| - 100% statement and branch coverage and complete public docstrings. | ||
|
|
||
| ## Documentation and changelog | ||
|
|
||
| Extend `docs/enterprise_issue_evidence_contracts.md` with the transient-text, | ||
| code-point-offset, privacy, normalization, and interpretation boundaries. Add an | ||
| authoritative changelog fragment and render `CHANGELOG.md` only after the final | ||
| fragment is stable. | ||
|
|
||
| ISO 8601-1:2019 remains the current published basic-rules standard as of this | ||
| plan; ISO has an Edition 2 committee draft under development. ISO 4217:2015 is | ||
| the current published currency-code standard and is maintained through its | ||
| registration authority. Python regular-expression match spans are Unicode string | ||
| indices, not user-perceived grapheme-cluster positions; therefore the contract | ||
| must state its offset unit explicitly rather than imply UAX #29 grapheme | ||
| segmentation. | ||
|
|
||
| ## References | ||
|
|
||
| International Organization for Standardization. (2015). *Codes for the | ||
| representation of currencies* (ISO Standard No. 4217:2015). | ||
| https://www.iso.org/standard/64758.html | ||
|
|
||
| International Organization for Standardization. (2019). *Date and time— | ||
| Representations for information interchange—Part 1: Basic rules* (ISO Standard | ||
| No. 8601-1:2019). https://www.iso.org/standard/70907.html | ||
|
|
||
| Python Software Foundation. (2025). *Python 3.13 standard library: `datetime`, | ||
| `decimal`, `hashlib`, `re`, and `typing`*. https://docs.python.org/3.13/ | ||
|
|
||
| The Unicode Consortium. (2025). *Unicode text segmentation* (Unicode Standard | ||
| Annex No. 29, Revision 47). https://www.unicode.org/reports/tr29/ |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.