Finance reconciliation: fixture pack (closes unimatrix27/ideas#24) - #2
Open
unimatrix27 wants to merge 1 commit into
Open
Finance reconciliation: fixture pack (closes unimatrix27/ideas#24)#2unimatrix27 wants to merge 1 commit into
unimatrix27 wants to merge 1 commit into
Conversation
One-shot, public-fork-safe fixture pack so NousResearch#22 (parsers + matcher) can be implemented fully offline. Ships: - tests/fixtures/finance/<vendor>/<invoice>.txt + .meta.json for Sipgate, Notion, Lucky Penny (invoice + paired credit note), and Vodafone. - tests/fixtures/finance/vodafone/portal_notification_*.txt — body of one notification-only email (Vodafone is portal-only most months). - tests/fixtures/finance/transactions.jsonl — 11 named TX ids + the Google Ads kanban-task row; counterparty IBANs redacted to "DE**". - tests/fixtures/finance/beleg_match_samples.jsonl — 9 rows incl. all 3 via='manual_review' shapes verbatim (load-bearing for NousResearch#20's backfill tests). - tests/fixtures/finance/belege_sent_samples.jsonl — 9 rows covering each via value, >=2 with bank_tx_id IS NULL, >=2 with attachments. - finance/scripts/build_fixtures.py + README — the re-runnable extractor. Re-running build_fixtures.py against the same Supabase + mailbox state produces byte-identical output. The script is NOT run in CI; it needs SUPABASE_DB_URL + the LINEO_MS_* delegated token bundle.
🔎 Lint report:
|
| Rule | Count |
|---|---|
PLW1514 |
1 |
First entries
finance/scripts/build_fixtures.py:138: [PLW1514] `pathlib.Path(...).read_text` without explicit `encoding` argument
✅ Fixed issues: none
Unchanged: 0 pre-existing issues carried over.
ty (type checker)
Total: 7972 on HEAD, 7967 on base (🆕 +5)
🆕 New issues (43):
| Rule | Count |
|---|---|
invalid-argument-type |
33 |
unresolved-import |
4 |
unresolved-attribute |
4 |
unsupported-operator |
1 |
possibly-missing-submodule |
1 |
First entries
run_agent.py:12752: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `str | Unknown | Divergent | ... omitted 3 union elements`
run_agent.py:5777: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["/"]` and `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:8745: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `Divergent | Unknown | str | ... omitted 3 union elements`
run_agent.py:4176: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:9324: [invalid-argument-type] invalid-argument-type: Argument to function `_get_anthropic_max_output` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:13287: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:12797: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:9489: [invalid-argument-type] invalid-argument-type: Argument to function `lmstudio_model_reasoning_options` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:12056: [invalid-argument-type] invalid-argument-type: Argument to function `apply_anthropic_cache_control` is incorrect: Expected `bool`, found `int | Divergent | Unknown | ... omitted 3 union elements`
run_agent.py:12799: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:11248: [invalid-argument-type] invalid-argument-type: Argument to function `_fixed_temperature_for_model` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
cli.py:8199: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:3347: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:8746: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `Divergent | Unknown | str | ... omitted 3 union elements`
run_agent.py:8746: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str | None`, found `Divergent | Unknown | str | ... omitted 3 union elements`
finance/tests/test_fixture_pack.py:11: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
run_agent.py:3347: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:13284: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown | str, Unknown | str | dict[str, str]] | Any | ... omitted 4 union elements`
run_agent.py:8665: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `str`, found `Divergent | Unknown | str | ... omitted 3 union elements`
run_agent.py:13518: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:12836: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:5313: [invalid-argument-type] invalid-argument-type: Argument to function `parse_rate_limit_headers` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:9341: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_profile` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:2501: [invalid-argument-type] invalid-argument-type: Argument to function `ensure_lmstudio_model_loaded` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
run_agent.py:5777: [unresolved-attribute] unresolved-attribute: Attribute `split` is not defined on `dict[Unknown, Unknown]`, `int`, `dict[Unknown | str, Unknown | str | dict[str, str]]` in union `str | Unknown | dict[Unknown, Unknown] | int | dict[Unknown | str, Unknown | str | dict[str, str]]`
... and 18 more
✅ Fixed issues (33):
| Rule | Count |
|---|---|
invalid-argument-type |
29 |
unresolved-attribute |
3 |
unsupported-operator |
1 |
First entries
cli.py:8199: [invalid-argument-type] invalid-argument-type: Argument to function `estimate_usage_cost` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:12056: [invalid-argument-type] invalid-argument-type: Argument to function `apply_anthropic_cache_control` is incorrect: Expected `bool`, found `int | str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | dict[Unknown, Unknown]`
run_agent.py:12836: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:5777: [unresolved-attribute] unresolved-attribute: Attribute `split` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]]`, `int`, `dict[Unknown, Unknown]` in union `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:12840: [invalid-argument-type] invalid-argument-type: Argument to bound method `SessionDB.update_token_counts` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:9324: [invalid-argument-type] invalid-argument-type: Argument to function `_get_anthropic_max_output` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:9516: [invalid-argument-type] invalid-argument-type: Argument to function `github_model_reasoning_efforts` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:8930: [invalid-argument-type] invalid-argument-type: Argument to function `get_transport` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13518: [invalid-argument-type] invalid-argument-type: Argument to function `_pool_may_recover_from_rate_limit` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:5777: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["/"]` and `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:8745: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:13738: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:3347: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_stale_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:5313: [invalid-argument-type] invalid-argument-type: Argument to function `parse_rate_limit_headers` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:12752: [invalid-argument-type] invalid-argument-type: Argument to function `normalize_usage` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:8746: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:8662: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `int`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:8746: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str | None`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:11319: [unresolved-attribute] unresolved-attribute: Attribute `strip` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `(str & ~AlwaysFalsy) | (Unknown & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:9768: [unresolved-attribute] unresolved-attribute: Attribute `lower` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `(str & ~AlwaysFalsy) | (Unknown & ~AlwaysFalsy) | (dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:13284: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
run_agent.py:13287: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:7160: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
run_agent.py:9341: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_profile` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
run_agent.py:2501: [invalid-argument-type] invalid-argument-type: Argument to function `ensure_lmstudio_model_loaded` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
... and 8 more
Unchanged: 4177 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Public-fork-safe fixture pack so
unimatrix27/ideas#22(parsers + matcher) can be implemented fully offline by a later session. One-shot bridge; the extraction script is committed but does not run in CI.Closes unimatrix27/ideas#24. Do not merge — Sebastian needs to eyeball the redactions before public-fork commit per the issue's hard constraint.
Files
finance/scripts/build_fixtures.pyfinance/scripts/README.mdfinance/tests/__init__.py,finance/tests/test_fixture_pack.pypytest hermes-agent/finance/tests/runstests/fixtures/finance/sipgate/B4373121.{txt,meta.json}tests/fixtures/finance/sipgate/B4411208.{txt,meta.json}tests/fixtures/finance/sipgate/B4459838.{txt,meta.json}tests/fixtures/finance/notion/ZWLWGPDN-0002.{txt,meta.json}tests/fixtures/finance/lucky_penny/6945-10683.{txt,meta.json}tests/fixtures/finance/lucky_penny/CN-6945-10021.{txt,meta.json}tests/fixtures/finance/vodafone/122203440401.{txt,meta.json}tests/fixtures/finance/vodafone/portal_notification_2026_04.{txt,meta.json}tests/fixtures/finance/transactions.jsonltests/fixtures/finance/beleg_match_samples.jsonltests/fixtures/finance/belege_sent_samples.jsonlviavalueRedaction summary — please eyeball before merging
The hard rule from NousResearch#24 is: real data, no synthesis, only redact what is unsafe for a public fork.
transactions.counterparty_iban"DE**"vodafone/122203440401.txt) carries a real mobile number0173/3749539(line 7). This is a vendor-issued SIM identifier shown on every Vodafone invoice but is also Sebastian's actual line. Treating it the same as Sipgate's product IDs because both NousResearch#22's parser and the matcher will key on it; flag here for an explicit go/no-go before public-fork commit.from,subject,internet_message_id)DE63XXXXXXXXXXXXXXXX66)beleg_match_samples/belege_sent_samplesbelege_sent_samplesrecipientcolumnbelege@datev.de-shaped addresses)Notes on the Vodafone notification body
The spec quoted the phrase "Ihre Rechnung liegt im Portal bereit". The actual recurring Vodafone notification text is:
— same semantic, different exact wording, with the email subject "Ihre Mobilfunk-Rechnung vom 14.04.2026 steht im Internet bereit." This is the real text from the recurring template; the spec's quote appears to be a paraphrase from memory. Honest fixture, no synthesis.
A Vodafone PDF was unexpectedly available in
rechnung@(received 2026-05-10, subject empty, X400-shaped sender — looks like a portal download forwarded inbound). I included that one too so NousResearch#22 has at least one Vodafone PDF text to parse; if you'd rather ship notification-only for the public fork, just deletevodafone/122203440401.{txt,meta.json}.Verification (each line corresponds to an acceptance criterion in NousResearch#24)
find tests/fixtures/finance -type flists exactly the paths above..txt, never the PDF binary. Confirmed — no.pdffiles in the diff.test_transactions_jsonl_has_named_idsenforces this infinance/tests/.via='manual_review'shapes present verbatim inbeleg_match_samples.jsonl(TX 37, 45, 83 — the last one carrieskanban_task='t_51751302'). Asserted bytest_beleg_match_samples_include_all_manual_review.belege_sentsample covers all 4viavalues, has ≥2 rows withbank_tx_id IS NULL, and ≥2 with non-emptyattachment_filenames. Asserted bytest_belege_sent_samples_cover_all_via(actual: 2 null, 7 with att).pytest hermes-agent/finance/tests/runs.python3 -m pytest finance/tests/ -q→21 passed in 0.72s.diff -rempty).finance/scripts/, not wired intopyproject.toml's default test addopts.set_session(readonly=True); Graph calls are allGET. No writes anywhere.Spec note worth a moment of review
NousResearch#24 says the extraction script should reuse
tools/microsoft_graph_client.py. That upstream client is wired for app-onlyclient_credentialsauth (MSGRAPH_TENANT_ID/MSGRAPH_CLIENT_SECRET), but the only access available is the operator's delegated token bundle in~/.hermes/lineo-ms-tokens/sebastian.json(refresh-token grant againstLINEO_MS_CLIENT_ID). To stay one-shot and avoid carrying an app-only secret on the EC2, the script does its own minimalurllibGETs against Graph and a smallDelegatedTokenCachefor the refresh flow. The shape of the data this script hands off to disk is identical to what the upstream client would have produced. Happy to refactor to aMicrosoftGraphClient(token_provider=DelegatedTokenCache())adapter if you'd prefer.Out of scope (intentionally not in this PR)
pdftextextraction path beyondpymupdf(nomarker-pdf— 3–5 GB dep is overkill for these fixtures, per Endless Terminals Environment Integration NousResearch/hermes-agent#24).🤖 Generated with Claude Code