Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions contributors/emails/muhammadfurqan0100@gmail.com
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
immuhammadfurqan
2 changes: 1 addition & 1 deletion optional-skills/mlops/pinecone/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: pinecone
description: Managed vector database for production AI applications. Fully managed, auto-scaling, with hybrid search (dense + sparse), metadata filtering, and namespaces. Low latency (<100ms p95). Use for production RAG, recommendation systems, or semantic search at scale. Best for serverless, managed infrastructure.
description: Managed vector DB for production RAG and search.
version: 1.0.0
author: Orchestra Research
license: MIT
Expand Down
108 changes: 108 additions & 0 deletions optional-skills/research/pinecone-research/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
---
name: pinecone-research
description: Agent RAG and long-term memory with Pinecone.
version: 1.0.0
author: immuhammadfurqan
license: MIT
dependencies: [pinecone-client, langchain-pinecone]
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [RAG, Pinecone, Memory, Research, Vector Database, Agent, Retrieval]

---

# Pinecone Research — Agent RAG & Long-Term Memory

Use Pinecone as a retrieval-augmented generation (RAG) backend for agent
conversations: persist embeddings, retrieve relevant context from past
sessions, and build long-term memory.

## When to use this skill

**Use when:**
- Building agent RAG pipelines with Pinecone as the vector store
- Need persistent long-term memory across agent sessions
- Combining retrieval with agent tool use
- Researching or prototyping semantic search workflows

**Use the mlops/pinecone skill instead when:**
- Need a general Pinecone reference (index management, CRUD, hybrid search)
- Working on production infrastructure without agent integration

## Quick start

### Setup

```bash
pip install pinecone-client langchain-pinecone langchain-openai
```

Set your API key:
```bash
export PINECONE_API_KEY="your-api-key"
```

### Basic RAG pipeline

```python
from pinecone import Pinecone, ServerlessSpec
from langchain_pinecone import PineconeVectorStore
from langchain_openai import OpenAIEmbeddings

# Initialize Pinecone
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

# Create or connect to index
index_name = "agent-memory"
if index_name not in [i.name for i in pc.list_indexes()]:
pc.create_index(
name=index_name,
dimension=1536,
metric="cosine",
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)

# Build vector store
vectorstore = PineconeVectorStore.from_documents(
documents=docs,
embedding=OpenAIEmbeddings(),
index_name=index_name,
)

# Retrieve relevant context
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
results = retriever.invoke("What did the agent discuss yesterday?")
```

### Namespace-based session memory

```python
# Store per-session memory
vectorstore = PineconeVectorStore(
index=pc.Index(index_name),
embedding=OpenAIEmbeddings(),
namespace=f"session-{session_id}",
)

# Query across all sessions (no namespace filter)
all_memory = PineconeVectorStore(
index=pc.Index(index_name),
embedding=OpenAIEmbeddings(),
)
results = all_memory.similarity_search("relevant query", k=10)
```

## Best practices

1. **Namespace by session or user** — isolate data for multi-tenant agents
2. **Batch upserts** — 100–200 vectors per batch for efficiency
3. **Metadata filtering** — tag vectors with session ID, timestamp, topic
4. **Prune old memory** — delete stale namespaces to control costs
5. **Use serverless** — auto-scaling, pay-per-use pricing

## Resources

- **Pinecone Docs**: https://docs.pinecone.io
- **LangChain Integration**: https://python.langchain.com/docs/integrations/vectorstores/pinecone
- **Free Tier**: 1 index, 100K vectors (1536 dimensions)
155 changes: 155 additions & 0 deletions optional-skills/research/pinecone-research/scripts/memory_manager.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
"""Pinecone memory manager — namespace-based session memory for agents.

Provides helpers for storing and retrieving agent conversation memory
using Pinecone namespaces. Each session gets its own namespace for isolation,
with cross-session search available via the global namespace.

Usage:
export PINECONE_API_KEY="your-key"
export OPENAI_API_KEY="your-key"
python memory_manager.py --index-name agent-memory --action store \
--session-id sess-001 --text "User discussed project architecture"
python memory_manager.py --index-name agent-memory --action recall \
--query "architecture decisions"
python memory_manager.py --index-name agent-memory --action cleanup \
--session-id sess-001
"""
from __future__ import annotations

import argparse
import hashlib
import os
import sys
import time


def get_pinecone_client():
"""Initialize Pinecone client from environment."""
try:
from pinecone import Pinecone
except ImportError:
print("Error: pinecone-client not installed. Run: pip install pinecone-client", file=sys.stderr)
sys.exit(1)

api_key = os.environ.get("PINECONE_API_KEY")
if not api_key:
print("Error: PINECONE_API_KEY environment variable not set.", file=sys.stderr)
sys.exit(1)

return Pinecone(api_key=api_key)


def get_embeddings():
"""Get the embedding model."""
try:
from langchain_openai import OpenAIEmbeddings
except ImportError:
print("Error: langchain-openai not installed. Run: pip install langchain-openai", file=sys.stderr)
sys.exit(1)
return OpenAIEmbeddings()


def store_memory(index, session_id: str, text: str, metadata: dict | None = None):
"""Store a memory entry in the session namespace."""
embeddings = get_embeddings()
vector = embeddings.embed_query(text)

doc_id = hashlib.sha256(f"{session_id}:{text}:{time.time()}".encode()).hexdigest()[:16]
entry_metadata = {
"text": text[:1000],
"session_id": session_id,
"timestamp": int(time.time()),
}
if metadata:
entry_metadata.update(metadata)

index.upsert(
vectors=[{"id": doc_id, "values": vector, "metadata": entry_metadata}],
namespace=session_id,
)
print(f"Stored memory [{doc_id}] in namespace '{session_id}'")
return doc_id


def recall_memories(index, query: str, session_id: str | None = None, top_k: int = 5):
"""Recall memories matching a query, optionally scoped to a session."""
embeddings = get_embeddings()
query_vector = embeddings.embed_query(query)

kwargs = {"vector": query_vector, "top_k": top_k, "include_metadata": True}
if session_id:
kwargs["namespace"] = session_id

results = index.query(**kwargs)

print(f"\nRecalling memories for: {query!r}")
if session_id:
print(f"Scoped to session: {session_id}")
print(f"Found {len(results['matches'])} results:\n")

for match in results["matches"]:
score = match["score"]
text = match["metadata"].get("text", "")[:200]
sess = match["metadata"].get("session_id", "unknown")
ts = match["metadata"].get("timestamp", 0)
print(f" [{score:.4f}] session={sess} time={ts}")
print(f" {text}")
print()

return results


def cleanup_session(index, session_id: str):
"""Delete all vectors in a session namespace."""
index.delete(delete_all=True, namespace=session_id)
print(f"Cleaned up namespace '{session_id}'")


def show_stats(index):
"""Show index statistics."""
stats = index.describe_index_stats()
print(f"Total vectors: {stats['total_vector_count']}")
namespaces = stats.get("namespaces", {})
if namespaces:
print(f"Namespaces ({len(namespaces)}):")
for ns, info in sorted(namespaces.items()):
print(f" '{ns}': {info['vector_count']} vectors")
else:
print("No namespaces found.")


def main():
parser = argparse.ArgumentParser(description="Pinecone agent memory manager")
parser.add_argument("--index-name", required=True, help="Pinecone index name")
parser.add_argument(
"--action",
choices=["store", "recall", "cleanup", "stats"],
required=True,
)
parser.add_argument("--session-id", help="Session namespace ID")
parser.add_argument("--text", help="Text to store as memory")
parser.add_argument("--query", help="Query for recall")
parser.add_argument("--top-k", type=int, default=5, help="Number of results")
args = parser.parse_args()

pc = get_pinecone_client()
index = pc.Index(args.index_name)

if args.action == "store":
if not args.session_id or not args.text:
parser.error("--session-id and --text required for store action")
store_memory(index, args.session_id, args.text)
elif args.action == "recall":
if not args.query:
parser.error("--query required for recall action")
recall_memories(index, args.query, session_id=args.session_id, top_k=args.top_k)
elif args.action == "cleanup":
if not args.session_id:
parser.error("--session-id required for cleanup action")
cleanup_session(index, args.session_id)
elif args.action == "stats":
show_stats(index)


if __name__ == "__main__":
main()
Loading
Loading