feat: add OpenCode SQLite session database support

[JakobSachs]

Played around with the project a little, but also wanted to carry over my Opencode sessions. The patch should hopefully be as minimal as possible.

Had to do this tmp-dir workaround to get the same directory-tree behaviour as with the other tools, trying not to touch too much of the existing logic. So it uses the dir-column in the DB to create a tmp-dir structure.
The table:

   session table (SQLite)
   ┌──────────┬─────────────────────┐
   │ id       │ directory           │
   ├──────────┼─────────────────────┤
   │ ses_a1b2 │ /home/user/frontend │
   │ ses_c3d4 │ /home/user/frontend │
   │ ses_e5f6 │ /home/user/backend  │
   │ ses_g7h8 │ /home/user/my-infra │
   └──────────┴─────────────────────┘

becomes

mempalace_oc_*/
├── frontend/
│   ├── frontend_ses_a1b2.txt
│   └── frontend_ses_c3d4.txt
├── backend/
│   └── backend_ses_e5f6.txt
└── my_infra/
    └── my_infra_ses_g7h8.txt

which is then handled the same way how claude-code/chatgpt/etc. ingestion works.

---

Ingest should work via:

mempalace mine ~/.local/share/opencode/opencode.db --mode convos

Tests + Lint should be clean

[thecodenomad]

Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10

[JakobSachs]

Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10

Using v1.3.13 on macOS, (mind you im not sure where opencode saves the db on other platforms..)

Did you get any error or just no-output/nothing found ?

[thecodenomad]

Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10

Using v1.3.13 on macOS, (mind you im not sure where opencode saves the db on other platforms..)

Did you get any error or just no-output/nothing found ?

No errors, it just didn't find any data, or at least it reported it found 0 files.

I'll update just to be sure and give it another go.

NOTE: On Linux I do see the db at ~/.local/share/opencode/opencode.db

.local/share/opencode
❯ ls
auth.json  log          opencode.db-shm  snapshot  tool-output
bin        opencode.db  opencode.db-wal  storage

[myInstagramAlternative]

opencode db path

[bensig]

Cool feature but conflicts with main. Can you rebase if you'd like to continue? We'd be interested in OpenCode support.

[JakobSachs]

Cool feature but conflicts with main. Can you rebase if you'd like to continue? We'd be interested in OpenCode support.

done, rebased 👍

[JakobSachs]

Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10

Using v1.3.13 on macOS, (mind you im not sure where opencode saves the db on other platforms..)
Did you get any error or just no-output/nothing found ?

No errors, it just didn't find any data, or at least it reported it found 0 files.

I'll update just to be sure and give it another go.

NOTE: On Linux I do see the db at ~/.local/share/opencode/opencode.db

.local/share/opencode
❯ ls
auth.json  log          opencode.db-shm  snapshot  tool-output
bin        opencode.db  opencode.db-wal  storage

Actually, found the issue with this: For some reason on my one Mac OpenCode has a different db path, on my private one it also uses the XDG data dir path (like on linux).

pushed a small fix to search both places so we catch all of it now 👍

[thecodenomad]

Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10

Using v1.3.13 on macOS, (mind you im not sure where opencode saves the db on other platforms..)
Did you get any error or just no-output/nothing found ?

No errors, it just didn't find any data, or at least it reported it found 0 files.
I'll update just to be sure and give it another go.
NOTE: On Linux I do see the db at ~/.local/share/opencode/opencode.db

.local/share/opencode
❯ ls
auth.json  log          opencode.db-shm  snapshot  tool-output
bin        opencode.db  opencode.db-wal  storage

Actually, found the issue with this: For some reason on my one Mac OpenCode has a different db path, on my private one it also uses the XDG data dir path (like on linux).

pushed a small fix to search both places so we catch all of it now 👍

Thank you! Just tried again, it worked for me! I did noticed that there was an unrelated failure:

Failed to send telemetry event CollectionGetEvent: capture() takes 1 positional argument but 3 were given

This appears to be due to a breaking change with posthog, but pinning it posthog = ">=2.4.0,<6.0.0" fixes it *(Opened #163 for that issue)

Nice work!

mining opencode convo

=======================================================
  MemPalace Mine — Conversations
=======================================================
  Wing:    step1
  Source:  /tmp/mempalace_oc_eszqwao7/step1
  Files:   4
  Palace:  /var/home/user/.local/distrobox_homes/cppdev/.mempalace/palace
-------------------------------------------------------

Before establishing client
Before client.get_collection
  ✓ [   1/4] step1_ses_330da6dbcffed2cb9JowsSAxJK.txt           +29
  ✓ [   2/4] step1_ses_335b2b72bffeS76Nj6gG2702Kw.txt           +11
  ✓ [   3/4] step1_ses_335bdf907ffeYLloh40zvEWBIl.txt           +4
  ✓ [   4/4] step1_ses_335dc12b1ffeZ1KP1hY8UDtjfv.txt           +8

=======================================================
  Done.
  Files processed: 4
  Files skipped (already filed): 0
  Drawers filed: 52

  By room:
    technical            3 files
    decisions            1 files

  Next: mempalace search "what you're looking for"
=======================================================

[bgauryy]

PR Review: feat: add OpenCode SQLite session database support

Executive Summary

Aspect	Value
PR Goal	Add OpenCode SQLite session database as a 6th supported conversation format for mining
Files Changed	7
Risk Level	🟡 MEDIUM - New I/O path with SQLite, has a Python 3.9 compat crash and a resource leak
Review Effort	3/5 - Moderate: new data source integration, SQL queries, temp-dir recursion pattern
Recommendation	🔄 REQUEST_CHANGES

Affected Areas: mempalace/normalize.py (core), mempalace/convo_miner.py (integration), tests/test_normalize.py, mempalace/cli.py, READMEs, uv.lock

Business Impact: Enables OpenCode users to mine their AI coding sessions into MemPalace — expands supported tool ecosystem.

Flow Changes: Adds an early-return path in mine_convos() for .db files that bypasses the normal scan_convos() → normalize() loop, creating temp directories grouped by project before recursing back into mine_convos().

Ratings

Aspect	Score
Correctness	3/5
Security	5/5
Performance	4/5
Maintainability	3/5

PR Health

• Has clear description
• References ticket/issue (not applicable — community feature contribution)
• Appropriate size (201 additions is reasonable)
• Has relevant tests (single happy-path test only)

---

High Priority Issues

🐛 #1: str | None union syntax crashes on Python 3.9

Location: mempalace/normalize.py — _resolve_opencode_db() and normalize_opencode_sessions() | Confidence: ✅ HIGH

The project declares requires-python = ">=3.9" in pyproject.toml and lists Python 3.9 in classifiers. The str | None union syntax for type hints is only available at runtime from Python 3.10+. On Python 3.9 this raises TypeError: unsupported operand type(s) for |: 'type' and 'NoneType' at import time, breaking the entire normalize module.

- def _resolve_opencode_db(db_path: str | None = None) -> str:
+ def _resolve_opencode_db(db_path: Optional[str] = None) -> str:

- def normalize_opencode_sessions(db_path: str | None = None) -> List[dict]:
+ def normalize_opencode_sessions(db_path: Optional[str] = None) -> List[dict]:

Optional is already imported in the file — this is a straightforward fix.

---

🐛 #2: SQLite connection leak on exception in main processing loop

Location: mempalace/normalize.py — normalize_opencode_sessions() | Confidence: ✅ HIGH

The SQLite connection is opened with conn = sqlite3.connect(resolved) and manually closed in two specific error branches (missing tables, no json_extract). However, if the main session-processing loop throws any exception (e.g., malformed JSON in json_extract, unexpected None in unpacking), conn.close() is never called. This leaks the connection and can leave the database file locked.

- def normalize_opencode_sessions(db_path: Optional[str] = None) -> List[dict]:
-     resolved = _resolve_opencode_db(db_path)
-     try:
-         conn = sqlite3.connect(resolved)
-         tables = { ... }
-     except Exception as e:
-         raise IOError(...)
-
-     if not {"session", "message", "part"}.issubset(tables):
-         conn.close()
-         raise IOError(...)
-     try:
-         conn.execute("SELECT json_extract('{}', '$')")
-     except Exception:
-         conn.close()
-         raise IOError(...)
-     sessions = conn.execute(...).fetchall()
-     results = []
-     for sid, title, directory, created in sessions:
-         ...  # ← exception here = connection leaked
-     conn.close()
-     return results
+ def normalize_opencode_sessions(db_path: Optional[str] = None) -> List[dict]:
+     resolved = _resolve_opencode_db(db_path)
+     try:
+         conn = sqlite3.connect(resolved)
+     except Exception as e:
+         raise IOError(f"Could not read SQLite database: {resolved}: {e}")
+
+     try:
+         tables = {
+             r[0] for r in conn.execute(
+                 "SELECT name FROM sqlite_master WHERE type='table'"
+             ).fetchall()
+         }
+         if not {"session", "message", "part"}.issubset(tables):
+             raise IOError(f"Not a recognized OpenCode database (missing tables): {resolved}")
+         conn.execute("SELECT json_extract('{}', '$')")
+
+         sessions = conn.execute(...).fetchall()
+         results = []
+         for sid, title, directory, created in sessions:
+             ...
+         return results
+     except IOError:
+         raise
+     except Exception as e:
+         raise IOError(f"Error reading OpenCode database: {resolved}: {e}")
+     finally:
+         conn.close()

---

Medium Priority Issues

🔄 #3: Dual processing path — .db files behave differently when found via directory scan

Location: mempalace/convo_miner.py:276 + mempalace/normalize.py:44 | Confidence: ✅ HIGH

Adding .db/.sqlite3/.sqlite to CONVO_EXTENSIONS means scan_convos() will discover these files during directory walks. However, the two discovery paths produce very different results:

Path	Trigger	Behavior
Direct .db file	mempalace mine opencode.db --mode convos	Early return → per-project temp dirs → grouped mining
.db in directory	mempalace mine ~/chats/ --mode convos	scan_convos() picks it up → normalize() → _normalize_opencode_sqlite() → ALL sessions concatenated into one blob → single wing/room

The directory-scan path loses project grouping and dumps all sessions into one flat transcript. Two options:

Option A — Remove .db/.sqlite3/.sqlite from CONVO_EXTENSIONS so only explicit file paths trigger OpenCode mining:

  CONVO_EXTENSIONS = {
      ".txt",
      ".md",
      ".json",
      ".jsonl",
-     ".db",
-     ".sqlite3",
-     ".sqlite",
  }

Option B — Handle .db files specially in the file-processing loop (check inside the for i, filepath in enumerate(files, 1) loop and delegate to the same temp-dir approach).

Option A is simpler and avoids surprising behavior. Users would need to point directly at the .db file, which is clearer.

---

🚨 #4: _extract_opencode_messages type hint mismatch

Location: mempalace/normalize.py — _extract_opencode_messages() | Confidence: ⚠️ MED

The parameter session_id: str = None declares type str but defaults to None. Should be Optional[str] = None for consistency with the type system and to avoid mypy/pyright warnings.

- def _extract_opencode_messages(
-     conn: sqlite3.Connection, session_id: str = None
- ) -> List[Tuple[str, str]]:
+ def _extract_opencode_messages(
+     conn: sqlite3.Connection, session_id: Optional[str] = None
+ ) -> List[Tuple[str, str]]:

---

🚨 #5: Error message uses input db_path instead of resolved path

Location: mempalace/normalize.py — normalize_opencode_sessions() | Confidence: ✅ HIGH

When the database is missing required tables, the error message prints the original db_path parameter (which could be None if called without arguments) instead of the resolved path:

- raise IOError(f"Not a recognized OpenCode database (missing tables): {db_path}")
+ raise IOError(f"Not a recognized OpenCode database (missing tables): {resolved}")

---

Low Priority Issues

🎨 #6: Missing space after # in comment

Location: mempalace/normalize.py — above OPENCODE_DB_PATHS | Confidence: ✅ HIGH

PEP 8 requires a space after # in inline comments.

- #List of known opencode db paths
+ # List of known opencode db paths

---

🎨 #7: Test coverage is thin — only happy path

Location: tests/test_normalize.py | Confidence: ⚠️ MED

The single test test_opencode_sqlite covers only the happy path (valid DB with one session). Missing test scenarios:

• Database without required tables (should raise IOError)
• Empty sessions (< 2 messages, should be skipped)
• Multiple sessions across different project directories
• The convo_miner.py temp-dir recursive mining flow
• json_extract unavailability error path
• Tool-call filtering (messages starting with "Called the ")

Not blocking, but the lack of error-path coverage means the resource leak (#2) could go unnoticed.

---

Flow Impact Analysis

User passes .db directly          User passes directory containing .db
          │                                       │
          ▼                                       ▼
  mine_convos()                            mine_convos()
  convo_path.is_file() ─── True            convo_path.is_file() ─── False
          │                                       │
          ▼                                       ▼
  normalize_opencode_sessions()             scan_convos() finds .db
  returns List[dict] per session            adds to files list
          │                                       │
          ▼                                       ▼
  Write to temp dirs by project             normalize() per file
  ┌── project_a/                                  │
  │   ├── session1.txt                            ▼
  │   └── session2.txt              _normalize_opencode_sqlite()
  └── project_b/                    concatenates ALL sessions
      └── session3.txt              into ONE transcript blob
          │                                       │
          ▼                                       ▼
  Recursive mine_convos()            chunk_exchanges() on blob
  per project subdir                 → single wing, single room
  → proper wing per project          → project grouping LOST

---

Created by Octocode MCP https://octocode.ai 🔍🐙

[bensig]

@JakobSachs pls see robot review above

[web3guru888]

✨ Review of #23 — feat: add OpenCode SQLite session database support

Scope: +201/−7 · 7 file(s)

• README.md (modified: +2/−2)
• mempalace/README.md (modified: +1/−1)
• mempalace/cli.py (modified: +1/−1)
• mempalace/convo_miner.py (modified: +34/−1)
• mempalace/normalize.py (modified: +123/−2)
• tests/test_normalize.py (modified: +30/−0)
• uv.lock (modified: +10/−0)

Issues

• ⚠️ Hardcoded filesystem path — breaks portability

Suggestions

• Magic number(s) 1000 — consider extracting to named constant(s)

Strengths

• ✅ Includes test coverage

🟡 Minor items — good work overall, a few things to address.

---

_{🏛️ Reviewed by MemPalace-AGI · Autonomous research system with perfect memory · Showcase: Truth Palace of Atlantis}

[igorls]

Hi, thanks for the contribution.

This PR has merge conflicts with develop, and the branch has not been updated in over 7 days, which puts it before our most recent release. The conflicts are likely against work that landed in that release.

Could you rebase onto develop so we can take another look?

If this change is no longer relevant, feel free to close the PR.

(This message is part of a periodic backlog pass, sent to all open PRs that match this state.)

[jphein]

Hi @JakobSachs — thanks for kicking off OpenCode support here, the DB-schema spadework (session / message / part with json_extract, the role-coerce and tool-echo / file-injection skips) is what gave me a clean read on the data when I picked this up.

I'm working in @jphein's fork on a version that targets the RFC 002 source-adapter contract (#990 / #989) — BaseSourceAdapter, declared transformations, describe_schema(), entry-point registration. The shape is different from #23, but the OpenCode-specific extraction logic is essentially what you wrote.

Three ways I could see this going, easiest to hardest from your side:

1. Co-authored — I cherry-pick the schema/extraction logic from your branch with Co-authored-by: on the commit, you get credit on the merge.
2. feat: add OpenCode SQLite session database support #23 lands first — you carry feat: add OpenCode SQLite session database support #23 over the rebase and merge as-is; I rework the RFC 002 version on top as a follow-up cleanup.
3. Something else — happy to defer to whatever you'd prefer.

No rush on this — I'm doing local prototyping in parallel so nothing is blocked on a response. Mostly want to make sure the coordination isn't a surprise.