feat(memory): verbatim transcript-chunk retrieval (flag-gated) - #7832
Conversation
Conversation vectors embed only the structured summary, so specific details (exact dates, names, numbers) are semantically unfindable. Chunk the raw transcript into overlapping dated windows and index them with the text in vector metadata; search returns verbatim evidence with zero extra reads.
Flag-gated (TRANSCRIPT_CHUNK_INDEXING_ENABLED, default off). Runs beside save_structured_vector on the postprocess executor.
Semantic search over raw transcript chunks — verbatim dated evidence to complement summary-level /conversations/search. LoCoMo eval: this layer moved the standard-protocol score 50.7% -> 89.3% (tuning) / 77.5% (held-out).
Transcripts are encrypted at rest in Firestore; storing chunk text as plaintext vector metadata would bypass that. Store only (conversation_id, chunk_index, created_at) and re-hydrate the verbatim text from Firestore at read time via deterministic re-chunking. Verified: search results byte-identical on a 20-question LoCoMo sample (20/20 same verdicts as the pre-refactor full run).
…ount deletion Conversation delete and the account-deletion purge now also remove ns_tchunks vectors (id-prefix listing). Verified live: upsert -> search finds 2 -> delete -> search finds 0.
Greptile SummaryThis PR adds a verbatim transcript-chunk retrieval layer over raw conversation transcripts. Conversation vectors in
Confidence Score: 3/5Safe to merge only after adding a flag guard to the search endpoint; the deletion-path overhead is minor but the endpoint leaking embedding costs in the default-off configuration contradicts the stated design intent. The search endpoint is always callable and always invokes
Important Files Changed
Sequence DiagramsequenceDiagram
participant PC as process_conversation
participant TC as transcript_chunks.py
participant VDB as vector_db (ns_tchunks)
participant OAI as OpenAI Embeddings
participant FS as Firestore
note over PC: TRANSCRIPT_CHUNK_INDEXING_ENABLED=true only
PC->>TC: build_transcript_chunks(segments, started_at)
TC-->>PC: "[{text, chunk_index, created_at}]"
PC->>VDB: upsert_transcript_chunk_vectors(uid, conv_id, chunks)
VDB->>OAI: embed_documents([chunk texts])
OAI-->>VDB: vectors[]
VDB->>VDB: upsert to Pinecone (metadata only, no text)
note over PC: Search path (always active)
PC->>VDB: search_transcript_chunks(uid, query, limit)
VDB->>OAI: embed_query(query)
OAI-->>VDB: query vector
VDB-->>PC: "[{conv_id, chunk_index, score}]"
PC->>TC: hydrate_chunk_texts(uid, rows)
TC->>FS: get_conversations_by_id(uid, conv_ids)
FS-->>TC: conversations (decrypted)
TC->>TC: rebuild chunks deterministically
TC-->>PC: "[{conv_id, chunk_index, score, text}]"
Reviews (1): Last reviewed commit: "feat(memory): delete transcript-chunk ve..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| """Semantic search over RAW transcript chunks (verbatim evidence with dates). | ||
|
|
||
| Complements /conversations/search, which matches against conversation summaries: | ||
| summaries drop specifics (exact dates, names, numbers), so detail questions need | ||
| this verbatim layer. Returns chunks newest-relevant with their conversation date. | ||
| """ | ||
| rows = vector_db.search_transcript_chunks(uid, body.query, limit=body.limit) | ||
| rows = hydrate_chunk_texts(uid, rows) | ||
| if not rows: | ||
| return _ok("search_conversation_chunks", f"No transcript excerpts found matching '{body.query}'.") | ||
| parts = [] | ||
| for i, r in enumerate(rows, 1): | ||
| parts.append(f"Excerpt {i} (relevance: {r['score']:.2f}):\n{r['text']}") | ||
| return _ok("search_conversation_chunks", "\n\n".join(parts)) | ||
|
|
||
|
|
||
| # --------------- memory endpoints --------------- | ||
|
|
||
|
|
There was a problem hiding this comment.
vector_db.search_transcript_chunks always calls embeddings.embed_query(query) (an OpenAI API call) when Pinecone is configured, regardless of TRANSCRIPT_CHUNK_INDEXING_ENABLED. With the default flag=off no chunks are ever indexed, so every call to this endpoint burns an embedding credit and returns an empty result. The PR description says merging this changes no production behavior, but the endpoint is live and callable the moment this ships.
A simple guard at the top of the handler, or inside search_transcript_chunks itself, would make this flag-consistent with the indexing path.
| logger.info(f'delete_conversation {conversation_id} {uid} cascade={cascade}') | ||
| conversations_db.delete_conversation(uid, conversation_id) | ||
| delete_vector(uid, conversation_id) | ||
| delete_transcript_chunk_vectors(uid, conversation_id) |
There was a problem hiding this comment.
list call on every conversation deletion
delete_transcript_chunk_vectors is called regardless of TRANSCRIPT_CHUNK_INDEXING_ENABLED. When the flag is off (default) and no chunks are indexed, index.list(prefix=..., namespace=TRANSCRIPT_CHUNKS_NAMESPACE) still makes a network round-trip to Pinecone on every single conversation deletion. The same pattern is repeated in users.py for account deletion, where it fires once per conversation ID in the user's history. The PR's "no production behavior change" claim is therefore inaccurate: every deletion in the current release incurs an extra Pinecone API call.
| for c in conversations: | ||
| segs = c.get('transcript_segments') or [] | ||
| started = c.get('started_at') or c.get('created_at') | ||
| chunks_by_conv[c['id']] = {ch['chunk_index']: ch['text'] for ch in build_transcript_chunks(segs, started)} |
There was a problem hiding this comment.
started_at type from Firestore dict is not validated before calling strftime
build_transcript_chunks expects started_at: Optional[datetime] and calls started_at.strftime(...) and started_at.timestamp(). Here started is read from a raw Firestore dict, and while the Python Firestore SDK normally returns DatetimeWithNanoseconds (a datetime subclass) for Timestamp fields, the guard only checks if started_at is not None. If any historical conversation stored the timestamp as an integer (Unix epoch) rather than a Firestore Timestamp, this would raise AttributeError: 'int' object has no attribute 'strftime' at hydration time, causing the chunk to be silently dropped.
1 participant
What
A verbatim retrieval layer over raw conversation transcripts. Conversation vectors (ns1) embed only the structured summary, so specific details — exact dates, names, numbers, one-off mentions — are semantically unfindable today. This adds:
ns_tchunks): raw transcript sliced into overlapping dated windows, embedded atprocess_conversationtime. Flag-gated:TRANSCRIPT_CHUNK_INDEXING_ENABLED, default OFF → merging this changes no production behavior.POST /v1/tools/conversations/search-chunks— semantic search returning verbatim dated excerpts.Why (measured)
Benchmarked with the industry-standard protocol (mem0's verbatim judge + answer prompts):
Reference points, same judge: Zep 58.4%, mem0 paper 66.9%, full-context baseline 72.9%, Letta 74%, mem0 Platform (vendor-reported) 92.5%.
Cost
Embeddings-only at ingestion (no LLM calls): ~$0.01/heavy-voice-user/month embedding + ~$0.07 Pinecone writes; storage accumulates to ~$0.10/user/mo after a year. ~0.2% of the same user's STT cost.
Verification
lint_async_blockers.py: clean. black: clean.Follow-ups (not in this PR)
search-chunksas a chat-agent tool (where the UX win lands) + re-verify abstention behavior.🤖 Generated with Claude Code