Skip to content

Add direct Turso database migration - #1923

Merged
stefan-burke merged 8 commits into
mainfrom
turso-migrate
Jul 26, 2026
Merged

Add direct Turso database migration#1923
stefan-burke merged 8 commits into
mainfrom
turso-migrate

Conversation

@stefan-burke

@stefan-burke stefan-burke commented Jul 25, 2026

Copy link
Copy Markdown
Member

Summary

  • Add deno task migrate:turso, which asks for the source database, the new Turso name, and any Turso details that are not already in .env.
  • Download one complete SQLite file, check it, and send that file through Turso's database upload API. The migration does not replay SQL statements one at a time.
  • Check that the new name is free before downloading. Remove an incomplete Turso database if setup, upload, or an interrupted run fails.
  • Print the new DB_URL and DB_TOKEN when the migration succeeds, even if temporary-file cleanup then needs operator help.

Safety

The task hides entered tokens, streams large files with an exact byte length, validates Turso responses before using them, and keeps temporary files owner-only. The app encryption key is not in the database file, so the task reminds the operator to keep using the source DB_ENCRYPTION_KEY.

Testing

  • nix develop -c deno task precommit
  • Turso client mutation test: 66 of 66 changes detected
  • Local wire contract verifies the received authorization, byte length, file bytes, and secure upload target
  • Snapshot interruption contract verifies that a stalled source sync closes and cleans up

Summary by CodeRabbit

  • New Features

    • Added an interactive workflow to migrate a remote database into a new Turso database via deno task migrate:turso.
    • Automatically validates the snapshot requirements for Turso upload and uses Turso’s native file upload flow, then outputs the new DB_URL and DB_TOKEN on success.
  • Documentation

    • Updated the tasks list and added TURSO_API_TOKEN, TURSO_ORGANIZATION, and TURSO_GROUP to the environment variable docs, including full migration and cleanup steps.
  • Bug Fixes

    • Improved cancellation/abort handling to stop snapshot sync/verification while still completing required cleanup of incomplete uploads.
  • Tests

    • Added/extended coverage for migration CLI behavior, uploader validation, snapshot abort safety, and termination-signal handling.

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 26 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 67419e13-7aa1-48cc-b9d8-88b77909981b

📥 Commits

Reviewing files that changed from the base of the PR and between 0366994 and ae7a1d5.

📒 Files selected for processing (2)
  • test/scripts/turso-migration.test.ts
  • test/test-utils/turso-migration.ts
📝 Walkthrough

Walkthrough

Adds an interactive deno task migrate:turso workflow that snapshots a source database, validates and uploads the SQLite file to a new Turso database, handles cancellation and cleanup, and documents the task and environment variables.

Changes

Turso migration and cancellation

Layer / File(s) Summary
Shared termination signal handling
scripts/termination-signals.ts, scripts/mutation/*, scripts/audit-equivalent-mutants.ts, test/scripts/*termination-signals*
Moves SIGINT/SIGTERM listener helpers into a shared module and updates callers and tests.
Snapshot cancellation and upload preparation
scripts/database-snapshot-lib.ts, scripts/turso-migration-file.ts, test/scripts/database-snapshot*.test.ts, test/scripts/turso-migration-file.test.ts
Adds abortable snapshot synchronization, configurable checks, SQLite PRAGMA validation, and authenticated streaming uploads.
Typed Turso API client
src/shared/turso-api.ts, test/integration/turso-api-management.test.ts, test/test-utils/turso-api.ts
Adds validated management operations, upload-seeded database creation, token generation, existence checks, listings, and cleanup.
Interactive migration orchestration
scripts/turso-migration*.ts, test/scripts/turso-migration.test.ts, test/test-utils/turso-migration.ts
Implements prompting, scope selection, snapshot migration, cleanup, credential output, and cancellation handling.
Task wiring and documentation
deno.json, AGENTS.md, README.md, test/shared/slug.test.ts
Exposes and documents the migration task, adjusts typecheck inputs, and removes a slug uniqueness test.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant MigrationCLI
  participant TursoAPI
  participant Snapshot
  participant Upload
  User->>MigrationCLI: Enter source and destination details
  MigrationCLI->>TursoAPI: Check destination and create upload-seeded database
  MigrationCLI->>Snapshot: Create and verify SQLite snapshot
  MigrationCLI->>Upload: Stream snapshot with database credentials
  Upload-->>MigrationCLI: Return upload response
  MigrationCLI-->>User: Print DB_URL and DB_TOKEN
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a direct migration workflow to Turso.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch turso-migrate
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch turso-migrate

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2365dd8454

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/turso-api.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f54752ae86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/turso-migration.ts
Comment thread scripts/turso-migration-lib.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e79bcdd180

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/turso-api.ts Outdated
Comment thread scripts/turso-migration-lib.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/turso-migration-lib.ts`:
- Around line 202-205: Update the environment callback passed to
readSnapshotRequest so it explicitly handles only DB_URL and the expected token
key, returning dbUrl or dbToken respectively; throw an error for any unexpected
key instead of defaulting to dbToken.

In `@test/scripts/termination-signals.test.ts`:
- Around line 9-24: Extend the termination-signal tests around
expectBothTerminationSignals to stub the listener registration method so it
throws, then invoke onTerminationSignals and offTerminationSignals and assert
both complete without throwing. Cover the fallback catch branch in
scripts/termination-signals.ts while preserving the existing successful-call
assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4a2c6676-9ef1-48dc-9f72-4a1e844928fe

📥 Commits

Reviewing files that changed from the base of the PR and between 67c03d8 and f0991a6.

📒 Files selected for processing (23)
  • AGENTS.md
  • README.md
  • deno.json
  • scripts/audit-equivalent-mutants.ts
  • scripts/database-snapshot-lib.ts
  • scripts/mutation/child-process.ts
  • scripts/mutation/isolation.ts
  • scripts/mutation/runner.ts
  • scripts/termination-signals.ts
  • scripts/turso-migration-file.ts
  • scripts/turso-migration-lib.ts
  • scripts/turso-migration.ts
  • src/shared/turso-api.ts
  • test/integration/turso-api-management.test.ts
  • test/scripts/database-snapshot-safety.test.ts
  • test/scripts/database-snapshot.test.ts
  • test/scripts/mutation-child-process.test.ts
  • test/scripts/termination-signals.test.ts
  • test/scripts/turso-migration-file.test.ts
  • test/scripts/turso-migration.test.ts
  • test/shared/slug.test.ts
  • test/test-utils/turso-api.ts
  • test/test-utils/turso-migration.ts
💤 Files with no reviewable changes (2)
  • test/shared/slug.test.ts
  • scripts/mutation/child-process.ts

Comment thread scripts/turso-migration-lib.ts
Comment thread test/scripts/termination-signals.test.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0991a6852

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/turso-api.ts Outdated
Comment thread scripts/database-snapshot-lib.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/shared/turso-api.ts (1)

196-201: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Clean up the database returned by the create response.

On a name mismatch, cleanup deletes the requested slug rather than database.Name. This can leave the created destination behind and, after a concurrent creation, delete an unrelated database with the requested name.

  • src/shared/turso-api.ts#L196-L201: pass database.Name to cleanUpFailedCreate in the mismatch branch.
  • test/integration/turso-api-management.test.ts#L208-L232: expect the DELETE request to target the returned database name.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/shared/turso-api.ts` around lines 196 - 201, Update the name-mismatch
branch in cleanUpFailedCreate usage within src/shared/turso-api.ts lines 196-201
to pass database.Name instead of the requested name. Update
test/integration/turso-api-management.test.ts lines 208-232 to expect the DELETE
request for the returned database name.
scripts/database-snapshot-lib.ts (1)

229-242: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Check abort state before starting the operation.

client.sync() is evaluated before waitForOrAbort calls throwIfAborted, so an already-aborted migration can still start a source sync.

  • scripts/database-snapshot-lib.ts#L229-L242: accept an operation thunk and call it only after checking signal.throwIfAborted().
  • test/scripts/database-snapshot-safety.test.ts#L125-L167: add a pre-aborted-signal test that asserts sync was never invoked and temporary files were removed.

As per coding guidelines, every bug fix must include a regression test reproducing the original defect.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/database-snapshot-lib.ts` around lines 229 - 242, Update
waitForOrAbort in scripts/database-snapshot-lib.ts:229-242 to accept an
operation thunk, check signal.throwIfAborted() first, and invoke the thunk only
afterward while preserving abort-race cleanup. Add a regression test in
test/scripts/database-snapshot-safety.test.ts:125-167 using a pre-aborted signal
that verifies sync is never invoked and temporary files are removed.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/scripts/termination-signals.test.ts`:
- Around line 34-46: The termination signal tests should verify per-signal
continuation rather than allowing a loop-wide catch to pass. Update the stubs in
the registration and removal tests to throw only for SIGINT, track calls to the
signal APIs, and assert that SIGTERM is still attempted after SIGINT fails in
both onTerminationSignals and offTerminationSignals.

---

Outside diff comments:
In `@scripts/database-snapshot-lib.ts`:
- Around line 229-242: Update waitForOrAbort in
scripts/database-snapshot-lib.ts:229-242 to accept an operation thunk, check
signal.throwIfAborted() first, and invoke the thunk only afterward while
preserving abort-race cleanup. Add a regression test in
test/scripts/database-snapshot-safety.test.ts:125-167 using a pre-aborted signal
that verifies sync is never invoked and temporary files are removed.

In `@src/shared/turso-api.ts`:
- Around line 196-201: Update the name-mismatch branch in cleanUpFailedCreate
usage within src/shared/turso-api.ts lines 196-201 to pass database.Name instead
of the requested name. Update test/integration/turso-api-management.test.ts
lines 208-232 to expect the DELETE request for the returned database name.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d11e2610-6436-476d-a79f-ebbd3354f7a0

📥 Commits

Reviewing files that changed from the base of the PR and between f0991a6 and 9382f63.

📒 Files selected for processing (6)
  • scripts/database-snapshot-lib.ts
  • scripts/turso-migration-lib.ts
  • src/shared/turso-api.ts
  • test/integration/turso-api-management.test.ts
  • test/scripts/database-snapshot-safety.test.ts
  • test/scripts/termination-signals.test.ts

Comment thread test/scripts/termination-signals.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9382f63cd5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/turso-api.ts
Comment thread scripts/turso-migration-lib.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e40231348

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/shared/turso-api.ts Outdated
Comment thread scripts/turso-migration-lib.ts Outdated
Comment thread scripts/turso-migration-file.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/test-utils/turso-migration.ts`:
- Around line 116-119: Update the verifyUploadFile test double in the migration
test utilities to record the provided AbortSignal in the relevant state, then
add a happy-path assertion that runMigrateTursoCli forwards state.deps.signal to
verification. Preserve the existing verification event recording and Promise
resolution behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 01bb09e4-695b-4ab3-83d5-9f9da715e2c6

📥 Commits

Reviewing files that changed from the base of the PR and between 9382f63 and 0366994.

📒 Files selected for processing (9)
  • scripts/database-snapshot-lib.ts
  • scripts/turso-migration-file.ts
  • scripts/turso-migration-lib.ts
  • scripts/turso-migration.ts
  • src/shared/turso-api.ts
  • test/integration/turso-api-management.test.ts
  • test/scripts/termination-signals.test.ts
  • test/scripts/turso-migration.test.ts
  • test/test-utils/turso-migration.ts

Comment thread test/test-utils/turso-migration.ts Outdated
@stefan-burke
stefan-burke added this pull request to the merge queue Jul 26, 2026
Merged via the queue into main with commit 865202f Jul 26, 2026
3 checks passed
@stefan-burke
stefan-burke deleted the turso-migrate branch July 26, 2026 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant