feat: add Gemini CLI session JSON normalizer - #155
Conversation
|
@bensig Same CI failure as PR #44 — not from this PR. Both failing tests are in
Pre-existing test-vs-code mismatch on |
de3bd94 to
c5669b9
Compare
Add _try_gemini_json parser for Gemini CLI session files stored at
~/.gemini/tmp/{project_hash}/chats/session-{timestamp}-{id}.json.
Gemini sessions are single JSON files (not JSONL) with a messages
array. User messages have type "user" with content as a list of
{"text": "..."} blocks (no "type" key — differs from Claude/OpenAI
content blocks). Assistant messages have type "gemini" with content
as a plain string.
Uses custom content extraction because Gemini content blocks omit
the "type" field that the shared _extract_content helper expects.
Fingerprints on "sessionId" + "messages" keys to avoid false
positives on other JSON formats.
Tested against real local Gemini CLI sessions. Session format
confirmed via Gemini CLI docs (session management, /resume command,
~/.gemini/tmp/{hash}/chats/ path).
Refs: MemPalace#59
c5669b9 to
44c9d2b
Compare
web3guru888
left a comment
There was a problem hiding this comment.
✨ Review of #155 — feat: add Gemini CLI session JSON normalizer
Scope: +47/−1 · 1 file(s)
mempalace/normalize.py(modified: +47/−1)
Suggestions
- 💡 No tests included — consider adding coverage for the new code paths
🟢 Approved — clean, well-structured PR. Good work @adv3nt3!
🏛️ Reviewed by MemPalace-AGI · Autonomous research system with perfect memory · Showcase: Truth Palace of Atlantis
|
Hi, thanks for the contribution. This PR has merge conflicts with Could you rebase onto If this change is no longer relevant, feel free to close the PR. (This message is part of a periodic backlog pass, sent to all open PRs that match this state.) |
|
Closing — superseded by #1234 (merged), which adds Gemini CLI session support via `_try_gemini_jsonl` against the actual JSONL session format (`~/.gemini/tmp/<project_hash>/chats/session-*.jsonl`, per google-gemini/gemini-cli#15292). This PR (#155) was written against an older single-JSON shape (`session-*.json` with `sessionId` + `messages` array) that doesn't match what Gemini CLI actually writes today. Milla's #1234 also landed a follow-up (#e7fe6ca) hardening detection so user/gemini turns appearing before the `session_metadata` sentinel are discarded — that's stronger than what was in this PR. No code worth porting. Thanks for the patient backlog ping. |
Summary
Add
_try_gemini_jsonparser for Gemini CLI session files stored at~/.gemini/tmp/{project_hash}/chats/session-{timestamp}-{id}.json. This is the 7th normalize format for MemPalace, alongside Claude AI JSON, ChatGPT JSON, Claude Code JSONL, Codex CLI JSONL (#61), Slack JSON, and plain text.Gemini CLI session format
Gemini CLI auto-saves every conversation as a single JSON file per session. Sessions are project-scoped — stored under a hash of the working directory. Retention defaults to 30 days / 100 sessions (configurable via
settings.json).Path:
~/.gemini/tmp/{project_hash}/chats/session-{timestamp}-{short_id}.jsonStructure:
{ "sessionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "projectHash": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...", "startTime": "2026-03-30T10:28:04.070Z", "lastUpdated": "2026-03-30T10:28:16.793Z", "messages": [ { "id": "xxxxxxxx-...", "timestamp": "2026-03-30T10:28:04.070Z", "type": "user", "content": [{"text": "Quick Terraform question about input validation..."}] }, { "id": "xxxxxxxx-...", "timestamp": "2026-03-30T10:28:16.793Z", "type": "gemini", "content": "Yes, the validation is **worth adding**..." } ], "kind": "main" }Message types
typevaluecontentformat"user"{"text": "..."}blocks"gemini"Other message types (model changes, tool calls, etc.) may appear in sessions but are skipped by this parser — only
userandgeminicarry conversation content.Design decisions
Custom content extraction instead of shared
_extract_contentGemini user content blocks are
{"text": "..."}without a"type"field. The shared_extract_contenthelper in normalize.py expects{"type": "text", "text": "..."}(the Claude/OpenAI convention) and returns empty string for Gemini blocks. Rather than modifying the shared helper (which could affect 5 other parsers),_try_gemini_jsondoes its own extraction:"text"key from each blockFingerprints on
sessionId+messageskeysThe parser requires both
sessionIdandmessagesin the top-level dict. This prevents false positives on:mappingkey, nosessionIdchat_messageswrapper, nosessionIdSingle JSON file (not JSONL)
Unlike Codex (JSONL per line) and Claude Code (JSONL per line), Gemini stores the entire session as one JSON object with a
messagesarray. This means the parser registers in the_try_normalize_jsondispatcher alongside the other JSON parsers (afterjson.loads), not in the JSONL section.What's NOT handled (and why)
/resume save <tag>for manual checkpoints and conversation forking. These may create additional session files. The parser handles them the same as regular sessions — if it hassessionId+messages, it normalizes.None(same threshold as all other parsers)."user"and"gemini"message types are extracted. Tool calls, model changes, and thinking level changes are skipped — they're operational metadata, not conversation content./chat shareexports: Gemini can export conversations to Markdown or JSON via/chat share. The exported JSON format may differ from the auto-saved session format. This parser targets auto-saved sessions only.Prior art
google-gemini/gemini-clilibrary)Changes
1 file changed (
mempalace/normalize.py), 47 insertions:_try_gemini_json()parser function with custom content extraction_try_normalize_json()dispatcher alongside other JSON parsersTest plan
ruff check mempalace/normalize.pypasses cleanruff format --checkalready formattedpython3 -m py_compile mempalace/normalize.pycompiles OK>marker transcriptsNonefor Claude AI JSON, ChatGPT JSON, Slack JSON, plain dict, empty dict, and list inputsRefs: #59