fix(lcm): structural FTS rebuilds log why + how long; undecryptable-row fixture (follow-up to #966) - #1022
Merged
Conversation
…ble-row fixture Follow-up to #966 (card t_d3963974). Today's two structural rebuilds (13 min + 6.5 min under _LOAD_LOCK) left no log line; the only trace was a PHASE=context_engine_load_slow WARNING minutes later. Now the decision site logs the reason (missing table / missing shadow / wrong column / schema-probe exception incl. transient SQLITE_BUSY) and an O(1) size hint before the rebuild, and the duration after it. Test fixture: one AEAD-prefixed row the (disabled) cipher cannot decrypt, so the write-lock test also gates the per-boot NULL-over-NULL FTS trigger write that #966 removed.
Kyzcreig
force-pushed
the
fix/lcm-undecryptable-row-fixture
branch
from
September 25, 2026 11:56
dafad80 to
30d8083
Compare
Collaborator
Author
|
🤖 merged-by: apollo · lane: boil-ocean-aged · gate: BYPASS: FR PAUSED by Ace ruling 2026-09-22 (state/fleetreview-pause-20260922.md); Apollo-reviewed lands via bypass · why: 0/1 commits on main by patch-id; content not in #1023 (no _fts_structural_problem on main); rebased clean onto 7ed45d9, re-authored to Kyzcreig noreply for check-attribution; 19 passed lcm fts/init_cost tests |
Kyzcreig
enabled auto-merge
September 25, 2026 11:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #966 (card t_d3963974, Apollo boot-cost freeze #3).
Why
The two structural FTS rebuilds on 2026-09-24 (05:07→05:20 = 13 min, 05:22→05:28 = 6.5 min, each under
_LOAD_LOCK+ the SQLite write lock, up to 12 turns queued) left no log line at all — the only trace wasPHASE=context_engine_load_slowminutes later, and py-spy.repair_external_content_ftsdecided toDROP+INSERT ... VALUES('rebuild')silently.What
repair_external_content_fts: when a structural rebuild is about to run, log why (missing table/missing shadow <name>/not fts5/column missing/schema-probe error: <exc>— the last one covers a transientSQLITE_BUSYon thesqlite_masterprobe, which the old code also answered with a rebuild) plus an O(1) size hint (max(rowid)of the content table), then log the duration after the rebuild commits._fts_needs_rebuild_structuralnow records its reason in_last_structural_reasonfor the caller.test_lcm_init_cost_regression.py::_fill): one AEAD-prefixed row the (disabled) test cipher cannot decrypt — the fleet DB has 3 such rows — sotest_engine_construction_needs_no_write_locknow also gates the per-boot NULL-over-NULLmsg_fts_updatetrigger write that fix(lcm): FTS parity COUNT(*) ran under _LOAD_LOCK on every engine load (freeze #3) #966 removed (4–57 s per boot on the fleet DB,3 row(s) undecryptable× 813 in the gateway log).Evidence
tests/context_engine/: 395 passed on this branch (/tmp/lcm3-suite2.txt).test_engine_construction_needs_no_write_lockfails atstore.py:_backfill_search_content(database is locked); GREEN here.No behavior change to the rebuild decision itself (that is #966); this is observability + a test gap.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.