fix(libsql): recover cancelled transactions and history migration

[BenKurrek]

Summary

• Fix conversation-history and timeline 503s caused by transcript-index migration racing a current message update.
• Prevent cancelled filesystem transactions from retaining the single libSQL writer lease in a detached rollback task.
• Move successful connection-checkout telemetry from DEBUG to TRACE, and emit accurate DEBUG diagnostics only for failed checkouts.
• QA evidence: deployment 047e9307-25b8-41fb-b943-795bfce08d2b first showed TimelineUnavailable alongside expected version 1, found version 2, then entered repeated 10-second writer checkout timeouts with queued=2. PR Collapse lifecycle state into the row-native process journal #6696 introduced both failing paths after PR fix(libsql): serialize writers and recover transient contention #6863 established the shared one-writer runtime.

Change Type

• Bug fix
• New feature
• Refactor
• Documentation
• CI/Infrastructure
• Security
• Dependencies

Linked Issue

Related #6871. Regression follow-up to #6696 and #6863.

Validation

• cargo fmt --all -- --check
• cargo clippy --all --benches --tests --examples --all-features -- -D warnings — Not run workspace-wide; affected crates passed cargo clippy -p ironclaw_libsql_runtime -p ironclaw_filesystem -p ironclaw_threads --all-targets --all-features -- -D warnings.
• cargo build — Not run separately; affected crates were compiled by tests and clippy.
• Relevant tests pass: full ironclaw_libsql_runtime, ironclaw_filesystem, and ironclaw_threads suites; focused storage architecture gate.
• cargo test --features integration if database-backed or integration behavior changed — Not applicable: the changed libSQL contract is covered against a real temporary libSQL database in the owning crate; no PostgreSQL or composition behavior changed.
• Manual testing — Live QA logs were used to reproduce and correlate the failure; this branch has not been deployed.
• If a coding agent was used and supports it, review-pr or pr-shepherd --fix was run before requesting review

Test Strategy

User behavior: Loading conversation history and timelines remains available while the one-time transcript migration overlaps an active message update; a cancelled storage transaction no longer leaves QA's writer lane unavailable until restart.

Risk areas:

• Model behavior
• Browser
• Side effect
• Persistence
• Security or permissions
• External provider
• Cross-component behavior

Tests added or updated:

• Unit or contract: filesystem_history_survives_transcript_migration_racing_a_message_update deterministically injects the observed CAS race through the real filesystem thread service.
• Reborn integration: Not applicable: both failures are owned by local thread-storage/runtime contracts and do not require turn orchestration.
• Recorded fixture: Not applicable: no model behavior changed.
• Browser E2E: Not applicable: no frontend behavior changed; the 503 source is covered below the HTTP presentation layer.
• Backend or runtime: dropping_storage_transaction_releases_writer_lane_synchronously uses a real local libSQL database and the production size-one writer runtime.
• Live canary: Not run; the branch is not deployed to QA.

What the tests prove:

• A message version change between the migration query and BEGIN IMMEDIATE no longer escapes as a history/timeline backend error.
• Dropping an active transaction rolls back uncommitted state, releases writer ownership synchronously, and permits the next write without a detached rollback retaining the only lease.
• Existing libSQL runtime, filesystem backend, transcript, and storage projection contracts remain green.

Commands run:

• cargo test -p ironclaw_libsql_runtime
• cargo test -p ironclaw_filesystem
• cargo test -p ironclaw_threads
• cargo fmt --all -- --check
• cargo clippy -p ironclaw_libsql_runtime -p ironclaw_filesystem -p ironclaw_threads --all-targets --all-features -- -D warnings
• cargo test -p ironclaw_architecture --test reborn_process_storage_scan_gate
• cargo test -p ironclaw_architecture — partially passed, then hit the pre-existing generated-WebUI ratchet failure for builtin.profile_set in committed crates/ironclaw_webui/frontend/dist assets; none of those files are changed here.

Security Impact

None. No permissions, network calls, secrets, file-access authority, tool execution, or sandbox policy changed. Runtime error displays remain redacted.

Reborn Trust-Boundary Checklist

• Public policy/evidence/trust-bearing types: Not applicable; no trust-bearing types were added or changed.
• Untrusted content enters prompts only through an envelope/escaping primitive. Not applicable; no prompt path changed.
• Hashes declare purpose; trust/binding/authenticity uses SHA-256/BLAKE3 or separate authenticity check. Not applicable; no hashes changed.
• New/changed status, exit, policy, runtime, or error variants: downstream match sites audited. Command/output: no variants changed; existing typed checkout errors are preserved.
• Security/durability serde(default) fields fail closed or have migration tests. Not applicable; no serialized fields changed.
• Queues/maps/buffers/counters have bounds and overflow-safe arithmetic. No queue or pool bounds changed.
• Driver/operator-visible errors have stable class semantics (Transient, Permanent, Misconfigured, PolicyDenied or equivalent). Existing checkout and filesystem error classification is unchanged.
• Sandbox/native/host names accurately describe trust boundary. Not applicable; no sandbox or host surface changed.

Database Impact

No schema or data migration. libSQL cancellation cleanup now discards an open transaction's connection so SQLite rolls it back on close and the pool creates a clean replacement. Transcript index migration re-reads each listed row inside its existing transaction; PostgreSQL behavior is otherwise unchanged.

Blast Radius

The changes touch the shared libSQL writer lease, libSQL filesystem transaction cancellation, and one-time transcript index migration. Main risks are connection-pool capacity recovery after discard and migration atomicity under concurrent writes; both have regression coverage plus the full owning-crate suites.

Rollback Plan

Revert commit 55033b0de and redeploy. There is no schema or persisted-format rollback. Reverting restores the known risk that a cancelled transaction can retain the sole writer lease and that a first-read migration race can surface a 503.

Review Follow-Through

Please focus review on deadpool's permanent object removal during cancellation and the transaction-local re-read in transcript migration. The broad architecture suite's unrelated generated-asset failure is documented above; the storage-specific architecture gate passes.

---

Review track: C

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[railway-app]

🚅 Deployed to the ironclaw-pr-6935 environment in ironclaw-ci-preview

Service	Status	Web	Updated (UTC)
ironclaw	✅ Success (View Logs)	Web	Jul 30, 2026 at 8:49 pm

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 74b9c025-5577-446f-8ea6-8306050f7324

📥 Commits

Reviewing files that changed from the base of the PR and between ae0989c and 55033b0.

📒 Files selected for processing (4)

• crates/ironclaw_filesystem/src/libsql.rs
• crates/ironclaw_libsql_runtime/src/lib.rs
• crates/ironclaw_threads/src/filesystem_service/thread_index.rs
• crates/ironclaw_threads/tests/filesystem_session_thread_contract.rs

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Improved transaction cleanup so uncommitted changes are rolled back immediately and write access is released reliably.
  • Prevented transcript migrations from overwriting newer message updates during concurrent activity.
  • Ensured migrated conversation history remains readable and preserves concurrent messages.
• Diagnostics
  • Improved logging for successful and failed write-connection acquisition.

Walkthrough

The change synchronously discards connections when transactions drop, separates writer-holder cleanup from connection disposal, and re-reads transcript rows inside migration transactions to avoid stale CAS updates. Regression tests cover rollback, writer reuse, and concurrent message updates.

Changes

Transaction and migration correctness

Layer / File(s)	Summary
Writer lease discard lifecycle crates/ironclaw_libsql_runtime/src/lib.rs	Write leases now use a separate holder guard. discard removes the pooled connection while the guard clears task ownership. Checkout logging now records success at trace level and failures at debug level.
Synchronous transaction disposal crates/ironclaw_filesystem/src/libsql.rs	Dropping an active transaction discards its connection instead of spawning asynchronous rollback work. Tests verify rollback and immediate writer-lane reuse.
Transcript migration row revalidation crates/ironclaw_threads/src/filesystem_service/thread_index.rs, crates/ironclaw_threads/tests/filesystem_session_thread_contract.rs	Migration re-fetches each row inside the transaction. Race-test infrastructure verifies that a concurrent message update remains readable.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MigrationRaceBackend
  participant ThreadIndexMigration
  participant LibSqlStorageTxn
  MigrationRaceBackend->>ThreadIndexMigration: return listed transcript row
  ThreadIndexMigration->>LibSqlStorageTxn: begin transaction
  MigrationRaceBackend->>MigrationRaceBackend: apply concurrent message update
  ThreadIndexMigration->>MigrationRaceBackend: re-fetch transcript row
  MigrationRaceBackend-->>ThreadIndexMigration: return current row version
  ThreadIndexMigration->>LibSqlStorageTxn: apply CAS migration update

Loading

Possibly related PRs

• nearai/ironclaw#5455: Related libSQL transaction and storage lifecycle changes.
• nearai/ironclaw#5751: Related transaction, connection lifecycle, and CAS handling changes.
• nearai/ironclaw#6096: Related concurrent filesystem message-write serialization and locking changes.

Suggested reviewers: ilblackdragon

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows Conventional Commits style and accurately describes transaction recovery and history migration fixes.
Description check	✅ Passed	The description covers all required sections, validation results, risks, database impact, rollback, and review focus; incomplete checks are explicitly explained.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Fix failing CI checks

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ironloopai]

🔎 Review · PR #6935

🟢 Completed · Review submitted

Submitted review →

Reviewed the complete trusted base-to-head comparison. The transaction discard, writer-holder lifecycle, checkout telemetry, migration re-read, and regression tests are coherent; no concrete actionable defects were found.

_{Automatic · PR opened · attempt 1 of 3 · completed in 1m 52s}

Run details

• Repository: nearai/ironclaw
• Base: main at ae0989c
• Head: codex/fix-libsql-qa-503 at 55033b0
• Created: Jul 30, 2026, 8:46 PM UTC
• Updated: Jul 30, 2026, 8:48 PM UTC
• Run: 1a7d586a-3245-4ec9-adaa-ca1020b05445
• Latest attempt: 1 · Completed · da663224-abd4-4ec3-86ad-0d44d04c7336

[ironloopai]

🔍 Review complete · PR #6935

✅ No actionable findings

Reviewed the complete trusted base-to-head comparison. The transaction discard, writer-holder lifecycle, checkout telemetry, migration re-read, and regression tests are coherent; no concrete actionable defects were found.

Validation and technical details

• Verified trusted refs resolve to base ae0989c and head 55033b0.
• Inspected all four changed files and surrounding transaction, pool recycling, pagination, migration, and test-double code.
• Confirmed the existing pool recycle hook rejects connections returned while a transaction is active, complementing the new explicit discard path.
• Ran git diff --check successfully.
• Focused cargo tests could not be rerun because cargo is unavailable in the review environment.
• Base: main
• Head: codex/fix-libsql-qa-503 at 55033b0
• Run: 1a7d586a-3245-4ec9-adaa-ca1020b05445

[github-actions]

Coverage ratchet

Ratchet mode: ENFORCING

RATCHET PASS: global
  observed: 86.45% (317678 / 367485 lines)
  floor:    85.54% (tolerance 0.5pp -> effective floor 85.04%)
  denominator: 367485 lines now vs 368757 at floor capture (-1272 lines, -0.34%) — not a material change

RATCHET PASS: ironclaw_runner
  observed: 87.13% (14946 / 17154 lines)
  floor:    86.87% (tolerance 0.5pp -> effective floor 86.37%)
  floor_covered_lines: 14669 (tolerance 20 lines -> effective floor 14649)
  denominator: 17154 lines now vs 16887 at floor capture (+267 lines, +1.58%) — not a material change

RATCHET PASS: ironclaw_processes
  observed: 88.96% (5898 / 6630 lines)
  floor:    88.07% (tolerance 0.5pp -> effective floor 87.57%)
  floor_covered_lines: 5839 (tolerance 20 lines -> effective floor 5819)
  denominator: 6630 lines now vs 6630 at floor capture (+0 lines, +0%) — not a material change

RATCHET PASS: ironclaw_turns
  observed: 87.43% (9796 / 11204 lines)
  floor:    86.21% (tolerance 0.5pp -> effective floor 85.71%)
  floor_covered_lines: 9453 (tolerance 20 lines -> effective floor 9433)
  denominator: 11204 lines now vs 10965 at floor capture (+239 lines, +2.18%) — not a material change

Reborn integration-tier coverage

Line coverage (Reborn crates): 86.45% — 317678 / 367485 lines

Per-crate breakdown (60 crates, lowest-covered first)

Crate	Line %	Covered / Total
ironclaw_memory	54.37%	635 / 1168
ironclaw_observability	64%	32 / 50
ironclaw_projects	70.61%	233 / 330
ironclaw_trust	73.87%	670 / 907
ironclaw_capabilities	74.52%	2854 / 3830
ironclaw_extractors	77.3%	538 / 696
ironclaw_filesystem	77.82%	5615 / 7215
ironclaw_reborn_cli	77.97%	10857 / 13924
ironclaw_wasm	78.22%	704 / 900
ironclaw_process_sandbox	80.75%	281 / 348
ironclaw_events	81.08%	1256 / 1549
ironclaw_llm	81.11%	22461 / 27692
ironclaw_auth	81.56%	6027 / 7390
ironclaw_memory_native	81.81%	2951 / 3607
ironclaw_first_party_extensions	82.58%	6760 / 8186
ironclaw_network	83.19%	891 / 1071
ironclaw_libsql_runtime	83.3%	384 / 461
ironclaw_host_api	84.01%	9842 / 11715
ironclaw_authorization	84.92%	715 / 842
ironclaw_extension_host	85.32%	21251 / 24907
ironclaw_operator	85.33%	5309 / 6222
ironclaw_reborn_event_store	85.51%	1222 / 1429
ironclaw_secrets	85.56%	2844 / 3324
ironclaw_hooks	86.18%	9915 / 11505
ironclaw_reborn_composition	86.21%	22103 / 25640
ironclaw_reborn_config	86.4%	2091 / 2420
ironclaw_scripts	86.42%	420 / 486
ironclaw_event_projections	86.51%	1372 / 1586
ironclaw_common	86.89%	1769 / 2036
ironclaw_runner	87.13%	14946 / 17154
ironclaw_skills	87.38%	4820 / 5516
ironclaw_turns	87.43%	9796 / 11204
ironclaw_extensions	87.67%	4779 / 5451
ironclaw_product	87.84%	22026 / 25074
ironclaw_webui	88.16%	11766 / 13346
ironclaw_reborn_traces	88.3%	11665 / 13211
ironclaw_slack_extension	88.3%	1887 / 2137
ironclaw_host_ingress	88.38%	639 / 723
ironclaw_threads	88.56%	4955 / 5595
ironclaw_processes	88.96%	5898 / 6630
ironclaw_host_runtime	89.11%	20655 / 23180
ironclaw_telegram_v2_adapter	89.35%	1434 / 1605
ironclaw_reborn_openai_compat	89.53%	3643 / 4069
ironclaw_reborn_identity	89.66%	451 / 503
ironclaw_approvals	89.81%	1816 / 2022
ironclaw_event_streams	90.19%	1048 / 1162
ironclaw_wasm_limiter	90.24%	74 / 82
ironclaw_loop_host	91%	17809 / 19570
ironclaw_resources	91.31%	4085 / 4474
ironclaw_conversations	91.42%	2387 / 2611
ironclaw_telegram_extension	92.16%	670 / 727
ironclaw_mcp	92.36%	1317 / 1426
ironclaw_agent_loop	94.21%	10406 / 11045
ironclaw_first_party_extension_ports	94.8%	3752 / 3958
ironclaw_outbound	94.83%	4054 / 4275
ironclaw_triggers	94.96%	3091 / 3255
ironclaw_prompt_envelope	95.52%	192 / 201
ironclaw_safety	95.58%	4152 / 4344
ironclaw_runtime_policy	97.29%	862 / 886
ironclaw_attachments	98.04%	601 / 613

This table itself is informational and never gates the PR on its own — not the percentage, not the per-crate holes, not the 0-coverage callout. A separate coverage ratchet (dry-run until enforce=true; see tests/integration/coverage-floor.toml) can fail the build on specific configured floors.

Exemptions (3 entry/entries excluded from the accounting above)

Module / Crate	Reason	Issue
crate: ironclaw_embeddings	v1-only: consumed only by root ironclaw (src/app.rs, src/tools/builtin/memory.rs, src/workspace/mod.rs, src/config/{mod,embeddings}.rs); no crates/* dependents. Covered by "Tests (Legacy)".	#5657
crate: ironclaw_gateway	v1-only: consumed only by root ironclaw (src/channels/web/platform/static_files.rs, src/channels/web/handlers/frontend.rs); no crates/* dependents. Covered by "Tests (Legacy)".	#5657
crate: ironclaw_tui	v1-only: consumed only by root ironclaw (src/main.rs, src/channels/tui.rs); no crates/* dependents. Crate's own doc comment confirms it bridges INTO v1, not Reborn. Covered by "Tests (Legacy)".	#5657

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.