feat/fix Re-enabled WAL with commit transaction management (Linux Verification Requested)

[tiensi]

Summary

There seems to be a WAL race condition within the builder.rs file where a session is created and get session is called immediately after, which in some instances can fail because the get_session call is looking at an old version of the sessions db + WAL from one thread before the create_session changes finishes propagating.

I believe the culprit to be the lack of an explicit commit transaction with WAL enabled. In the concurrency section of sqlite documentation, When a read operation begins on a WAL-mode database, it first remembers the location of the last valid commit record in the WAL. So even though we were relying on concurrency through await?; create_session never explicitly called commit, which possibly resulted in get_session "misses" on an old version of the database.

This could also explain why the Pragma wal_checkpoint approach didn't work as the checkpoint didn't have a completed commit to apply WAL file changes to the database.

Type of Change

• Feature
• Bug fix
• Refactor / Code quality
• Performance improvement
• Documentation
• Tests
• Security fix
• Build / Release
• Other (specify below)

AI Assistance

• This PR was created or reviewed with AI assistance

Testing

I was unable to reproduce on multiple linux docker images, I went ahead and reproduced the "bug" (create_session and get_session race condition) by writing concurrent create_session -> get_session race condition tests and ran them a few thousand times.

Ran a timing check on an existing concurrency test. time cargo test test_concurrent_session_creation --release -- --test-threads=1;

Eyeballing the results, WAL takes about .763s and without takes .848s, about a 10% improvement.

Related Issues

Relates to #5197
Discussion:

• Previous fix

[Copilot]

Pull Request Overview

This PR re-enables SQLite WAL (Write-Ahead Logging) mode with explicit transaction management to fix a race condition where get_session could read stale data immediately after create_session. The solution adds explicit tx.commit() calls to ensure WAL changes are visible to concurrent readers.

Key changes:

• Enabled WAL journal mode in SQLite connection options
• Added explicit transaction management with commit calls to write operations
• Ensures WAL checkpoints occur at transaction boundaries for consistent reads

[Copilot]

The transaction commits before calling replace_conversation, breaking atomicity of the import. If replace_conversation fails, the session will be imported without messages. Consider including the conversation import in the same transaction, or document why this split is intentional.

Suggested change

	tx.commit().await?;
	if let Some(conversation) = &session.conversation {
	self.replace_conversation(&session.id, conversation).await?;
	}
	if let Some(conversation) = &session.conversation {
	self.replace_conversation_tx(&session.id, conversation, &mut tx).await?;
	}
	tx.commit().await?;

[tiensi]

I was on the fence with changing this, replace_conversation already does it's own commit independent of import_session. Easiest solution would be to unwrap replace_conversation inside of import_session.

If we move towards foreign keys then messages should not be session orphaned by default and can be addressed there.

[jamadeo]

Fantastic, thank you @tiensi . I was able to test this on @DOsinga 's machine, the same environment that suffered the original bug, and this does indeed fix it.