Skip to content

fix(db): harden migration recovery snapshots - #12435

Merged
diegosouzapw merged 4 commits into
release/v3.8.51from
fix/v3851-migration-151-152-safety
Sep 4, 2026
Merged

diegosouzapw merged 4 commits into
release/v3.8.51from
fix/v3851-migration-151-152-safety

Conversation

@diegosouzapw

@diegosouzapw diegosouzapw commented Sep 2, 2026 •

Copy link
Copy Markdown
Owner

Summary

Hardens SQLite upgrades around the historical migration 074 slot collision and makes every
actionable migration of a database that physically existed before initialization depend on a
durable safety snapshot.

  • validates that the required physical tables exist before accepting migration markers; when a
    table is missing, resolves the known 074_discovery_results / 081_inspector_custom_hosts
    provenance and replays the legitimate migration atomically
  • separates physical pre-existence from the logical fresh-seed state, so a pre-existing setup
    skeleton is snapshotted without reintroducing the false mass-migration abort on first setup
  • publishes content-addressed snapshots with fsync and atomic, no-overwrite hard links
  • canonicalizes sql.js exports in a detached VACUUM clone so identical retries reuse one snapshot
  • keeps repair planning and the mass-migration barrier under an IMMEDIATE writer transaction
  • removes migration-window pruning and keeps normal/manual retention as the cleanup boundary
  • extends the DATA_DIR guard to direct Node --eval / --print test probes
  • splits migration-runner helpers into focused modules so the production file-size ratchet passes
  • documents that DISABLE_SQLITE_AUTO_BACKUP never bypasses migration safety

TDD regressions

Case RED evidence GREEN evidence
Pre-existing setup skeleton must snapshot before migration 4/5; expected snapshot failure was not raised 5/5; fail-closed snapshot and successful setup-path snapshot both proven
Marker 081 without physical inspector table must replay inner suite 11/13; two recovery-count assertions failed inner suite 13/13; outer isolated wrapper 1/1

Final validation

Check Result Evidence
Focused DB and DATA_DIR matrix PASS 40/40 across the six changed/relevant unit files; every runtime used a fresh /tmp DATA_DIR
Fresh-setup integration path PASS first serve advances without the mass-migration abort; exactly one snapshot contains the pre-existing provider row
Snapshot fail-closed regression PASS pre-existing setup skeleton remains unmodified when snapshot publication fails
TypeScript core PASS npm run typecheck:core, exit 0
API route typecheck PASS 289 diagnostics, all within the frozen pre-existing baseline; no new API type error
Focused official ESLint PASS all changed TypeScript files with the repository suppression baseline, exit 0
Anti test-masking PASS five modified tests; zero removed/weakened assertions, skips, deletions, or tautologies
File-size ratchet PASS migrationRunner.ts 1170/1201; core.ts 1744/1745
Dependency cycles PASS no cycles across 422 files
Repository integrity gates PASS tracked-artifacts, changelog integrity, and migration numbering
Documentation and formatting PASS docs-all, Prettier, and git diff --check; 87 stale-version docs warnings remain informational
Independent static review PASS no material findings; extraction and the two safety fixes preserve the shared transaction contracts
Full repository lint INTERRUPTED earlier full run was CPU-bound after 26 minutes and exited 130; focused official ESLint is green, but this is not represented as a full-lint pass
Full release matrix HOLD not run by this scoped PR
Current GitHub checks PASS terminal draft fast-path on head b2661074f0: 9 success, 0 failure, 0 pending, 7 intentionally skipped, 2 neutral

Reconciliation evidence

Item Exact SHA / evidence
Original PR head received dbbf20dd9d57af8f0b287cfc83a2d9f640def118
Previous documented release base 97041954171ee3aa3f51a7fb01748797c23716f8
Exact release base selected for this reconciliation bf0d902dfc5369bd025f64808d96fe6cb473ea75
Normal merge commit, no rebase/force bff879a4e87b5984edb72567ae40430462374f2f
Final candidate head b2661074f02d671b1e5973b7f21e41cdfa2bd58b
Original PR delta across the base merge patch-id preserved as 78ba4e1288b2e91c242b3a91eb83add34d11e797 before the additional reviewed hardening commit

Operational contract and residual scope

Filesystems without same-filesystem hard links or durable file synchronization fail closed before
an existing persistent database is migrated. Windows directory-entry fsync remains best effort,
and sql.js snapshot canonicalization temporarily holds multiple full database images in memory.
These are explicit availability/resource tradeoffs, not silent safety bypasses.

The new DATA_DIR guard covers direct Node test/eval/print probes. Arbitrary script-file execution
such as node --import tsx file.ts is not claimed as closed by this PR and remains follow-up scope.
The new 081 cases exercise the shared atomic replay path; fault injection during the specific
081 DDL is not duplicated because the same transaction rollback is already covered through the
074 replay regression.

State: OWN SCOPE PASS · REMOTE DRAFT FAST-PATH PASS · FULL RELEASE HOLD

@diegosouzapw
diegosouzapw marked this pull request as ready for review September 4, 2026 00:00
@diegosouzapw
diegosouzapw merged commit 7ae8bf4 into release/v3.8.51 Sep 4, 2026
21 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
Validado em lote numa worktree combinada com os 14 PRs desta campanha de error-boundary sobre o tip de `release/v3.8.51`: `typecheck:core` limpo e **120/120** nos 23 arquivos de teste que os PRs trazem.

Um ponto que só apareceu no tree combinado: **diegosouzapw#12465 e diegosouzapw#12466 criam o mesmo arquivo novo** `open-sse/utils/streamReadiness.ts` (que não existe no tip) com desenhos divergentes de cancelamento — `cancelled` + `releaseLock` imediato num, `readInFlight`/`cancelRequested` com `cancelReader` fire-and-forget no outro. Adotei a versão do diegosouzapw#12466, que difere e defere o release do lock para quando a leitura em voo termina, e validei a escolha rodando as suítes dos **dois** PRs contra ela: 21/21 no readiness compartilhado e 22/22 incluindo o boundary do Perplexity.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant