Skip to content

fix(search): stop extreme relative date offsets from crashing recall (#3217) - #3413

Merged
nicoloboschi merged 1 commit into
mainfrom
fix/temporal-analysis-extreme-offsets-3217
Aug 12, 2026
Merged

fix(search): stop extreme relative date offsets from crashing recall (#3217)#3413
nicoloboschi merged 1 commit into
mainfrom
fix/temporal-analysis-extreme-offsets-3217

Conversation

@nicoloboschi

Copy link
Copy Markdown
Collaborator

Fixes #3217.

Root cause — query-time arithmetic, not stored rows

The three observed failures (year -534, year -974, year -97974) are exactly now.year − {2560, 3000, 100000}. They come from "N years ago" offset arithmetic at query-analysis time, not from dates stored in the DB:

  • Consolidation recalls with stored fact text as the query. A fact mentioning e.g. 十万年前 (100,000 years ago) is re-analyzed on every recall, so the failure is deterministic per bank and retries never help.
  • extract_period() runs before analyze()'s dateparser guard (fix(query_analyzer): handle dateparser internal crashes gracefully #893), so ValueError('year N is out of range') / OverflowError escaped straight out of recall as Failed to search memories (...), which is what killed consolidation.
  • No pathological row can exist in the date columns: Python datetime cannot represent pre-year-1 dates, so nothing outside 1–9999 can reach PostgreSQL through asyncpg in the first place (asyncpg decodes BC values with a different error — OverflowError: date value out of range — and -97974 is outside even PostgreSQL's representable range). This confirms @koriyoshi2041's analysis that the retain-time clamp proposed in the issue targets a non-existent ingress — no retain changes here.

#2636 already fixed this exact class for add_years (which is why the 年前 trio no longer reproduces on main), but sibling paths were left unguarded. All of these crash recall on current main:

'三万个月前的事件'  -> ValueError: year -474 is out of range   (add_months)
'过去三万个月'      -> ValueError: year -474 is out of range   (subtract_months)
'一百万天前的历史'  -> OverflowError: date value out of range  (raw timedelta days)
'九十九万周前'      -> OverflowError: date value out of range  (raw timedelta weeks)
'june 0000'         -> ValueError: year 0 is out of range      (non-Chinese month+year)

Fix — three layers

  1. extract_temporal_constraint() (the recall choke point) now degrades any analyzer failure to "no temporal signal" with a warning. analyze() itself stays strict, so parser bugs still surface in tests and to direct callers (test_query_analyzer_period_valueerror_still_surfaces is unchanged). This is the guarantee that one pathological phrase can never poison a bank's recall/consolidation again.
  2. chinese_temporal_periods: add_months/subtract_months are bounds-checked exactly like fix(search): avoid year-0 crashes in Chinese rolling-window temporal extraction #2636's add_years (return None, plumbed through every call site — ty enforces completeness), and day/week offsets go through the overflow-guarded add_days instead of raw timedelta addition. subtract_months collapses into add_months(reference_date, -months) (they were the same formula).
  3. temporal_periods: an explicit month + year 0000 match returns NO_TEMPORAL_CONSTRAINT instead of crashing datetime() (and instead of letting the dateparser fallback invent a different date).

Tests

tests/test_query_analyzer.py: 450 passed (409 existing + new).

Note on #3403

#3403 attributes the crash to datetime.fromisoformat() on stored ISO strings, but fromisoformat raises Invalid isoformat string (not year N is out of range) for negative-year input, _as_dt runs after recall so its errors can't produce the observed Failed to search memories (...) wrapper, and clamping to naive datetime(1, 1, 1) would introduce naive-vs-aware TypeErrors in _merge_min/_merge_max. That path isn't the reported bug.

…3217)

Consolidation recalls with stored fact text as the query, so a phrase
like "十万年前" (100,000 years ago) hit unguarded offset arithmetic in
extract_period — which runs BEFORE analyze()'s dateparser guard — and
escaped as ValueError('year -97974 is out of range'), deterministically
failing every recall and consolidation touching the bank. The three
years observed in #3217 (-534, -974, -97974) are exactly
now.year - {2560, 3000, 100000}: query-time arithmetic, not stored rows
(Python datetimes can't represent them, so no bad date can reach the DB
through asyncpg in the first place).

Three layers, mirroring the #2636 add_years fix:

- extract_temporal_constraint (the recall choke point) degrades any
  analyzer failure to 'no temporal signal' with a warning; analyze()
  itself stays strict so parser bugs still surface in tests.
- chinese_temporal_periods: add_months/subtract_months are now
  bounds-checked like add_years (returning None, plumbed through every
  call site), and day/week offsets go through the overflow-guarded
  add_days instead of raw timedelta addition.
- temporal_periods: an explicit month + year 0000 match returns
  NO_TEMPORAL_CONSTRAINT instead of crashing datetime().
@nicoloboschi
nicoloboschi merged commit 423e25d into main Aug 12, 2026
213 of 214 checks passed
@nicoloboschi
nicoloboschi deleted the fix/temporal-analysis-extreme-offsets-3217 branch August 12, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search: extreme historical dates in stored facts crash recall with 'year N is out of range'

1 participant