diff --git a/.agent/aidevops/claude-flow-comparison.md b/.agent/aidevops/claude-flow-comparison.md new file mode 100644 index 000000000..7b5a89a2f --- /dev/null +++ b/.agent/aidevops/claude-flow-comparison.md @@ -0,0 +1,81 @@ +--- +description: Feature comparison between aidevops and Claude-Flow +mode: subagent +tools: + read: true + write: false + edit: false + bash: false + glob: false + grep: false +model: haiku +--- + +# Claude-Flow vs aidevops Comparison + +Selective feature adoption from [ruvnet/claude-flow](https://github.com/ruvnet/claude-flow) v3. + +## Philosophy + +| Aspect | Claude-Flow | aidevops | +|--------|-------------|----------| +| Language | TypeScript (~340MB) | Shell scripts (~2MB) | +| Dependencies | Heavy (ONNX, WASM, gRPC) | Minimal (sqlite3, curl) | +| Architecture | Monolithic orchestrator | Composable subagents | +| Model routing | Automatic 3-tier | Guided via frontmatter | +| Memory | HNSW vector (built-in) | FTS5 default + embeddings opt-in | +| Coordination | Byzantine fault-tolerant | Async TOON mailbox | + +## Feature Adoption Status + +| Feature | Claude-Flow | aidevops Adoption | Status | +|---------|-------------|-------------------|--------| +| Vector memory | HNSW (built-in) | Optional embeddings via all-MiniLM-L6-v2 | Done | +| Cost routing | 3-tier automatic (SONA) | Model tier guidance + `model:` frontmatter | Done | +| Self-learning | SONA neural architecture | SUCCESS/FAILURE pattern tracking | Done | +| Swarm consensus | Byzantine/Raft | Skipped (async mailbox sufficient) | Skipped | +| WASM transforms | Agent Booster | Skipped (Edit tool fast enough) | Skipped | + +## What We Adopted + +### 1. Cost-Aware Model Routing + +**Claude-Flow**: Automatic 3-tier routing with SONA neural architecture that learns optimal model selection. + +**aidevops**: Documented routing guidance in `tools/context/model-routing.md` with `model:` field in subagent YAML frontmatter. Five tiers: haiku, flash, sonnet, pro, opus. `/route` command suggests optimal tier for a task. + +**Why this approach**: aidevops agents run inside existing AI tools (Claude Code, OpenCode, Cursor) which handle model selection. Guidance is more appropriate than automatic routing. + +### 2. Semantic Memory with Embeddings + +**Claude-Flow**: Built-in HNSW vector index, always-on semantic search. + +**aidevops**: Optional `memory-embeddings-helper.sh` using all-MiniLM-L6-v2 (~90MB). FTS5 keyword search remains default. `--semantic` flag on `memory-helper.sh recall` delegates to embeddings when available. + +**Why this approach**: Most memory queries work fine with keyword search. Embeddings add ~90MB of dependencies. Opt-in keeps the framework lightweight for users who don't need it. + +### 3. Success Pattern Tracking + +**Claude-Flow**: SONA neural architecture tracks routing decisions and outcomes. + +**aidevops**: `pattern-tracker-helper.sh` records SUCCESS_PATTERN and FAILURE_PATTERN memories tagged with task type and model tier. `/patterns` command surfaces relevant patterns for new tasks. + +**Why this approach**: Simple pattern storage in existing SQLite memory is sufficient. No need for neural architecture when the pattern corpus is small (hundreds, not millions). + +## What We Skipped + +### Swarm Consensus + +**Claude-Flow**: Byzantine fault-tolerant coordination for multi-agent consensus. + +**Why skipped**: aidevops uses async TOON mailbox for inter-agent communication. Most tasks don't need consensus - they need coordination. The mailbox pattern is simpler and sufficient. + +### WASM Transforms (Agent Booster) + +**Claude-Flow**: WASM-based code transforms for performance. + +**Why skipped**: The Edit tool is already fast enough. WASM adds complexity without meaningful benefit for the file sizes aidevops typically handles. + +## Key Insight + +Claude-Flow solves problems at scale (thousands of agents, millions of memories, real-time routing). aidevops operates at human scale (1-5 agents, hundreds of memories, session-based routing). The right solution depends on the scale. diff --git a/.agent/memory/README.md b/.agent/memory/README.md index cbac7bdce..b0e726a8f 100644 --- a/.agent/memory/README.md +++ b/.agent/memory/README.md @@ -84,6 +84,8 @@ See `scripts/commands/remember.md` and `scripts/commands/recall.md` for full doc | `TOOL_CONFIG` | Tool setup notes | | `DECISION` | Architecture decisions | | `CONTEXT` | Background info | +| `SUCCESS_PATTERN` | Approaches that consistently work for task types | +| `FAILURE_PATTERN` | Approaches that consistently fail for task types | ## Relation Types @@ -105,11 +107,53 @@ Inspired by Supermemory's relational versioning: This enables temporal reasoning like "what happened last week?" by distinguishing between when you learned something vs when it happened. +## Semantic Search (Opt-in) + +For similarity-based search beyond keyword matching, enable vector embeddings: + +```bash +# One-time setup (~90MB model download) +memory-embeddings-helper.sh setup + +# Index existing memories +memory-embeddings-helper.sh index + +# Search by meaning (not just keywords) +memory-helper.sh recall "how to optimize database queries" --semantic + +# Or use the embeddings helper directly +memory-embeddings-helper.sh search "authentication patterns" +``` + +FTS5 keyword search remains the default. Semantic search requires Python 3.9+ and sentence-transformers. + +## Pattern Tracking + +Track what works and what fails across task types and models: + +```bash +# Record a pattern +pattern-tracker-helper.sh record --outcome success --task-type bugfix \ + --model sonnet --description "Structured debugging found root cause" + +# Get suggestions for a task +pattern-tracker-helper.sh suggest "refactor the auth middleware" + +# View pattern statistics +pattern-tracker-helper.sh stats + +# Or use the /patterns command +/patterns refactor +``` + +See `scripts/pattern-tracker-helper.sh` for full documentation. + ## Storage Location ```text ~/.aidevops/.agent-workspace/memory/ ├── memory.db # SQLite database with FTS5 +├── embeddings.db # Optional: vector embeddings for semantic search └── preferences/ # Optional: markdown preference files ``` diff --git a/.agent/scripts/commands/patterns.md b/.agent/scripts/commands/patterns.md new file mode 100644 index 000000000..e4967291b --- /dev/null +++ b/.agent/scripts/commands/patterns.md @@ -0,0 +1,41 @@ +--- +description: Show success/failure patterns from memory to guide task approach +agent: Build+ +mode: subagent +model: haiku +--- + +Analyze and display success/failure patterns relevant to the current context. + +Arguments: $ARGUMENTS + +## Instructions + +1. If arguments are provided, use them as a task description to find relevant patterns: + +```bash +~/.aidevops/agents/scripts/pattern-tracker-helper.sh suggest "$ARGUMENTS" +``` + +2. If no arguments, show overall pattern statistics and recent patterns: + +```bash +~/.aidevops/agents/scripts/pattern-tracker-helper.sh stats +~/.aidevops/agents/scripts/pattern-tracker-helper.sh analyze --limit 5 +``` + +3. Present the results with actionable guidance: + - Highlight what approaches have worked for similar tasks + - Warn about approaches that have failed + - Suggest the optimal model tier based on pattern data + +4. If no patterns exist yet, explain how to start recording: + +```text +No patterns recorded yet. Patterns are recorded automatically during +development loops, or manually with: + + pattern-tracker-helper.sh record --outcome success \ + --task-type bugfix --model sonnet \ + --description "Structured debugging approach found root cause quickly" +``` diff --git a/.agent/scripts/commands/route.md b/.agent/scripts/commands/route.md new file mode 100644 index 000000000..f203356f8 --- /dev/null +++ b/.agent/scripts/commands/route.md @@ -0,0 +1,38 @@ +--- +description: Suggest optimal model tier for a task description +agent: Build+ +mode: subagent +model: haiku +--- + +Analyze the task description and recommend the optimal model tier. + +Task: $ARGUMENTS + +## Instructions + +1. Read `tools/context/model-routing.md` for the routing rules and tier definitions. + +2. Analyze the task description against the routing rules: + - **Complexity**: Simple transform vs reasoning vs novel design + - **Context size**: Small focused task vs large codebase sweep + - **Output type**: Classification vs code vs architecture + +3. Output a recommendation in this format: + +```text +Recommended: {tier} ({model_name}) +Reason: {one-line justification} +Cost: ~{relative}x vs sonnet baseline +``` + +4. If the task is ambiguous, suggest the tier and note what would push it up or down: + +```text +Recommended: sonnet (claude-sonnet-4) +Reason: Code modification with moderate reasoning +Cost: ~1x baseline + +Could be haiku if: the change is a simple rename/reformat +Could be opus if: the change requires architectural decisions +``` diff --git a/.agent/scripts/memory-embeddings-helper.sh b/.agent/scripts/memory-embeddings-helper.sh new file mode 100755 index 000000000..89daba229 --- /dev/null +++ b/.agent/scripts/memory-embeddings-helper.sh @@ -0,0 +1,575 @@ +#!/usr/bin/env bash +# memory-embeddings-helper.sh - Semantic memory search using vector embeddings +# Opt-in enhancement for memory-helper.sh (FTS5 remains the default) +# +# Uses all-MiniLM-L6-v2 (~90MB) for embeddings, SQLite for vector storage. +# Requires: Python 3.9+, sentence-transformers, numpy +# +# Usage: +# memory-embeddings-helper.sh setup # Install dependencies + download model +# memory-embeddings-helper.sh index # Index all existing memories +# memory-embeddings-helper.sh search "query" # Semantic similarity search +# memory-embeddings-helper.sh search "query" --limit 10 +# memory-embeddings-helper.sh add # Add single memory to index +# memory-embeddings-helper.sh status # Show index stats +# memory-embeddings-helper.sh rebuild # Rebuild entire index +# memory-embeddings-helper.sh help # Show this help + +set -euo pipefail + +# Configuration +readonly MEMORY_DIR="${AIDEVOPS_MEMORY_DIR:-$HOME/.aidevops/.agent-workspace/memory}" +readonly MEMORY_DB="$MEMORY_DIR/memory.db" +readonly EMBEDDINGS_DB="$MEMORY_DIR/embeddings.db" +readonly MODEL_NAME="all-MiniLM-L6-v2" +readonly EMBEDDING_DIM=384 +readonly PYTHON_SCRIPT="$MEMORY_DIR/.embeddings-engine.py" + +# Colors +readonly RED='\033[0;31m' +readonly GREEN='\033[0;32m' +readonly YELLOW='\033[0;33m' +readonly BLUE='\033[0;34m' +readonly NC='\033[0m' + +log_info() { echo -e "${BLUE}[INFO]${NC} $*"; } +log_success() { echo -e "${GREEN}[OK]${NC} $*"; } +log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; } +log_error() { echo -e "${RED}[ERROR]${NC} $*" >&2; } + +####################################### +# Check if dependencies are installed +####################################### +check_deps() { + local missing=() + + if ! command -v python3 &>/dev/null; then + missing+=("python3") + fi + + if ! python3 -c "import sentence_transformers" &>/dev/null 2>&1; then + missing+=("sentence-transformers") + fi + + if ! python3 -c "import numpy" &>/dev/null 2>&1; then + missing+=("numpy") + fi + + if [[ ${#missing[@]} -gt 0 ]]; then + return 1 + fi + return 0 +} + +####################################### +# Print missing dependency instructions +####################################### +print_setup_instructions() { + log_error "Missing dependencies for semantic memory." + echo "" + echo "Install with:" + echo " pip install sentence-transformers numpy" + echo "" + echo "Or run:" + echo " memory-embeddings-helper.sh setup" + echo "" + echo "This is opt-in. FTS5 keyword search (memory-helper.sh recall) works without this." + return 1 +} + +####################################### +# Create the Python embedding engine +# Kept as a single file for simplicity +####################################### +create_python_engine() { + mkdir -p "$MEMORY_DIR" + cat > "$PYTHON_SCRIPT" << 'PYEOF' +#!/usr/bin/env python3 +"""Embedding engine for aidevops semantic memory. + +Commands: + embed - Output embedding as JSON array + search [limit] - Search embeddings DB for similar + index - Index all memories + add - Index single memory + status - Show index stats +""" + +import json +import sqlite3 +import struct +import sys +from pathlib import Path + +import numpy as np + +# Lazy-load model to avoid slow imports on every call +_model = None + +def get_model(): + global _model + if _model is None: + from sentence_transformers import SentenceTransformer + _model = SentenceTransformer("all-MiniLM-L6-v2") + return _model + +def embed_text(text: str) -> list[float]: + model = get_model() + embedding = model.encode(text, normalize_embeddings=True) + return embedding.tolist() + +def pack_embedding(embedding: list[float]) -> bytes: + return struct.pack(f"{len(embedding)}f", *embedding) + +def unpack_embedding(data: bytes, dim: int = 384) -> list[float]: + return list(struct.unpack(f"{dim}f", data)) + +def cosine_similarity(a: list[float], b: list[float]) -> float: + a_arr = np.array(a) + b_arr = np.array(b) + dot = np.dot(a_arr, b_arr) + norm_a = np.linalg.norm(a_arr) + norm_b = np.linalg.norm(b_arr) + if norm_a == 0 or norm_b == 0: + return 0.0 + return float(dot / (norm_a * norm_b)) + +def init_embeddings_db(db_path: str): + conn = sqlite3.connect(db_path) + conn.execute(""" + CREATE TABLE IF NOT EXISTS embeddings ( + memory_id TEXT PRIMARY KEY, + embedding BLOB NOT NULL, + content_hash TEXT NOT NULL, + indexed_at TEXT DEFAULT CURRENT_TIMESTAMP + ) + """) + conn.commit() + return conn + +def cmd_embed(text: str): + embedding = embed_text(text) + print(json.dumps(embedding)) + +def cmd_search(embeddings_db: str, memory_db: str, query: str, limit: int = 5): + query_embedding = embed_text(query) + + conn = init_embeddings_db(embeddings_db) + rows = conn.execute("SELECT memory_id, embedding FROM embeddings").fetchall() + conn.close() + + if not rows: + print(json.dumps([])) + return + + results = [] + for memory_id, emb_blob in rows: + stored_embedding = unpack_embedding(emb_blob) + score = cosine_similarity(query_embedding, stored_embedding) + results.append((memory_id, score)) + + results.sort(key=lambda x: x[1], reverse=True) + top_results = results[:limit] + + # Fetch memory content for top results + mem_conn = sqlite3.connect(memory_db) + output = [] + for memory_id, score in top_results: + row = mem_conn.execute( + "SELECT content, type, tags, confidence, created_at FROM learnings WHERE id = ?", + (memory_id,) + ).fetchone() + if row: + output.append({ + "id": memory_id, + "content": row[0], + "type": row[1], + "tags": row[2], + "confidence": row[3], + "created_at": row[4], + "score": round(score, 4), + }) + mem_conn.close() + print(json.dumps(output)) + +def cmd_index(memory_db: str, embeddings_db: str): + mem_conn = sqlite3.connect(memory_db) + rows = mem_conn.execute("SELECT id, content, type, tags FROM learnings").fetchall() + mem_conn.close() + + if not rows: + print(json.dumps({"indexed": 0, "skipped": 0})) + return + + emb_conn = init_embeddings_db(embeddings_db) + existing = set( + r[0] for r in emb_conn.execute("SELECT memory_id FROM embeddings").fetchall() + ) + + indexed = 0 + skipped = 0 + for memory_id, content, mem_type, tags in rows: + import hashlib + content_hash = hashlib.md5(content.encode()).hexdigest() + + # Check if already indexed with same content + existing_hash = emb_conn.execute( + "SELECT content_hash FROM embeddings WHERE memory_id = ?", + (memory_id,) + ).fetchone() + + if existing_hash and existing_hash[0] == content_hash: + skipped += 1 + continue + + # Combine content with type and tags for richer embedding + combined = f"[{mem_type}] {content}" + if tags: + combined += f" (tags: {tags})" + + embedding = embed_text(combined) + packed = pack_embedding(embedding) + + emb_conn.execute( + """INSERT OR REPLACE INTO embeddings (memory_id, embedding, content_hash) + VALUES (?, ?, ?)""", + (memory_id, packed, content_hash) + ) + indexed += 1 + + emb_conn.commit() + emb_conn.close() + print(json.dumps({"indexed": indexed, "skipped": skipped, "total": len(rows)})) + +def cmd_add(memory_db: str, embeddings_db: str, memory_id: str): + mem_conn = sqlite3.connect(memory_db) + row = mem_conn.execute( + "SELECT content, type, tags FROM learnings WHERE id = ?", + (memory_id,) + ).fetchone() + mem_conn.close() + + if not row: + print(json.dumps({"error": f"Memory {memory_id} not found"})) + sys.exit(1) + + content, mem_type, tags = row + import hashlib + content_hash = hashlib.md5(content.encode()).hexdigest() + + combined = f"[{mem_type}] {content}" + if tags: + combined += f" (tags: {tags})" + + embedding = embed_text(combined) + packed = pack_embedding(embedding) + + emb_conn = init_embeddings_db(embeddings_db) + emb_conn.execute( + """INSERT OR REPLACE INTO embeddings (memory_id, embedding, content_hash) + VALUES (?, ?, ?)""", + (memory_id, packed, content_hash) + ) + emb_conn.commit() + emb_conn.close() + print(json.dumps({"indexed": memory_id})) + +def cmd_status(embeddings_db: str): + db_path = Path(embeddings_db) + if not db_path.exists(): + print(json.dumps({"exists": False, "count": 0, "size_mb": 0})) + return + + conn = sqlite3.connect(embeddings_db) + count = conn.execute("SELECT COUNT(*) FROM embeddings").fetchone()[0] + conn.close() + + size_mb = round(db_path.stat().st_size / (1024 * 1024), 2) + print(json.dumps({"exists": True, "count": count, "size_mb": size_mb})) + +if __name__ == "__main__": + if len(sys.argv) < 2: + print(__doc__) + sys.exit(1) + + command = sys.argv[1] + + if command == "embed": + cmd_embed(sys.argv[2]) + elif command == "search": + cmd_search(sys.argv[2], sys.argv[3], sys.argv[4], int(sys.argv[5]) if len(sys.argv) > 5 else 5) + elif command == "index": + cmd_index(sys.argv[2], sys.argv[3]) + elif command == "add": + cmd_add(sys.argv[2], sys.argv[3], sys.argv[4]) + elif command == "status": + cmd_status(sys.argv[2]) + else: + print(f"Unknown command: {command}") + sys.exit(1) +PYEOF + chmod +x "$PYTHON_SCRIPT" + return 0 +} + +####################################### +# Setup: install dependencies and model +####################################### +cmd_setup() { + log_info "Setting up semantic memory embeddings..." + + if ! command -v python3 &>/dev/null; then + log_error "Python 3 is required. Install it first." + return 1 + fi + + log_info "Installing Python dependencies..." + pip install --quiet sentence-transformers numpy + + log_info "Creating embedding engine..." + create_python_engine + + log_info "Downloading model (all-MiniLM-L6-v2, ~90MB)..." + python3 "$PYTHON_SCRIPT" embed "test" > /dev/null + + log_success "Semantic memory setup complete." + log_info "Run 'memory-embeddings-helper.sh index' to index existing memories." + return 0 +} + +####################################### +# Index all existing memories +####################################### +cmd_index() { + if ! check_deps; then + print_setup_instructions + return 1 + fi + + if [[ ! -f "$MEMORY_DB" ]]; then + log_error "Memory database not found at $MEMORY_DB" + log_error "Store some memories first with: memory-helper.sh store --content \"...\"" + return 1 + fi + + create_python_engine + + log_info "Indexing memories with $MODEL_NAME..." + local result + result=$(python3 "$PYTHON_SCRIPT" index "$MEMORY_DB" "$EMBEDDINGS_DB") + + local indexed skipped total + if command -v jq &>/dev/null; then + indexed=$(echo "$result" | jq -r '.indexed') + skipped=$(echo "$result" | jq -r '.skipped') + total=$(echo "$result" | jq -r '.total') + else + indexed=$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['indexed'])") + skipped=$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['skipped'])") + total=$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['total'])") + fi + + log_success "Indexed $indexed new memories ($skipped unchanged, $total total)" + return 0 +} + +####################################### +# Search memories semantically +####################################### +cmd_search() { + local query="" + local limit=5 + local format="text" + + while [[ $# -gt 0 ]]; do + case "$1" in + --limit|-l) limit="$2"; shift 2 ;; + --json) format="json"; shift ;; + --format) format="$2"; shift 2 ;; + *) + if [[ -z "$query" ]]; then + query="$1" + fi + shift + ;; + esac + done + + if [[ -z "$query" ]]; then + log_error "Query is required: memory-embeddings-helper.sh search \"your query\"" + return 1 + fi + + if ! check_deps; then + print_setup_instructions + return 1 + fi + + if [[ ! -f "$EMBEDDINGS_DB" ]]; then + log_error "Embeddings index not found. Run: memory-embeddings-helper.sh index" + return 1 + fi + + create_python_engine + + local result + result=$(python3 "$PYTHON_SCRIPT" search "$EMBEDDINGS_DB" "$MEMORY_DB" "$query" "$limit") + + if [[ "$format" == "json" ]]; then + echo "$result" + else + echo "" + echo "=== Semantic Search: \"$query\" ===" + echo "" + if command -v jq &>/dev/null; then + echo "$result" | jq -r '.[] | "[\(.type)] (score: \(.score)) \(.confidence)\n \(.content)\n Tags: \(.tags // "none")\n Created: \(.created_at)\n"' + else + python3 -c " +import json, sys +results = json.loads(sys.stdin.read()) +for r in results: + print(f'[{r[\"type\"]}] (score: {r[\"score\"]}) {r[\"confidence\"]}') + print(f' {r[\"content\"]}') + print(f' Tags: {r.get(\"tags\", \"none\")}') + print(f' Created: {r[\"created_at\"]}') + print() +" <<< "$result" + fi + fi + return 0 +} + +####################################### +# Add single memory to index +####################################### +cmd_add() { + local memory_id="$1" + + if [[ -z "$memory_id" ]]; then + log_error "Memory ID required: memory-embeddings-helper.sh add " + return 1 + fi + + if ! check_deps; then + print_setup_instructions + return 1 + fi + + create_python_engine + + local result + result=$(python3 "$PYTHON_SCRIPT" add "$MEMORY_DB" "$EMBEDDINGS_DB" "$memory_id") + + if echo "$result" | grep -q '"error"'; then + log_error "$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['error'])" 2>/dev/null || echo "$result")" + return 1 + fi + + log_success "Indexed memory: $memory_id" + return 0 +} + +####################################### +# Show index status +####################################### +cmd_status() { + if [[ ! -f "$EMBEDDINGS_DB" ]]; then + log_info "Embeddings index: not created" + log_info "Run 'memory-embeddings-helper.sh setup' to enable semantic search" + return 0 + fi + + if ! check_deps; then + log_warn "Dependencies not installed but index exists" + log_info "Run 'memory-embeddings-helper.sh setup' to install dependencies" + return 0 + fi + + create_python_engine + + local result + result=$(python3 "$PYTHON_SCRIPT" status "$EMBEDDINGS_DB") + + local count size_mb + if command -v jq &>/dev/null; then + count=$(echo "$result" | jq -r '.count') + size_mb=$(echo "$result" | jq -r '.size_mb') + else + count=$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['count'])") + size_mb=$(echo "$result" | python3 -c "import sys,json; print(json.load(sys.stdin)['size_mb'])") + fi + + log_info "Embeddings index: $count memories indexed (${size_mb}MB)" + log_info "Model: $MODEL_NAME (${EMBEDDING_DIM}d)" + log_info "Database: $EMBEDDINGS_DB" + + # Compare with memory DB + if [[ -f "$MEMORY_DB" ]]; then + local total_memories + total_memories=$(sqlite3 "$MEMORY_DB" "SELECT COUNT(*) FROM learnings;" 2>/dev/null || echo "?") + log_info "Total memories: $total_memories ($(( total_memories - count )) unindexed)" + fi + return 0 +} + +####################################### +# Rebuild entire index +####################################### +cmd_rebuild() { + log_info "Rebuilding embeddings index..." + + if [[ -f "$EMBEDDINGS_DB" ]]; then + rm "$EMBEDDINGS_DB" + log_info "Removed old index" + fi + + cmd_index + return 0 +} + +####################################### +# Show help +####################################### +cmd_help() { + echo "memory-embeddings-helper.sh - Semantic memory search (opt-in)" + echo "" + echo "Usage:" + echo " memory-embeddings-helper.sh setup Install dependencies + model" + echo " memory-embeddings-helper.sh index Index all existing memories" + echo " memory-embeddings-helper.sh search \"query\" Semantic similarity search" + echo " memory-embeddings-helper.sh search \"q\" -l 10 Search with custom limit" + echo " memory-embeddings-helper.sh add Index single memory" + echo " memory-embeddings-helper.sh status Show index stats" + echo " memory-embeddings-helper.sh rebuild Rebuild entire index" + echo " memory-embeddings-helper.sh help Show this help" + echo "" + echo "This is opt-in. FTS5 keyword search (memory-helper.sh recall) works without this." + echo "Requires: Python 3.9+, sentence-transformers (~90MB model download)" + return 0 +} + +####################################### +# Main entry point +####################################### +main() { + local command="${1:-help}" + shift || true + + case "$command" in + setup) cmd_setup ;; + index) cmd_index ;; + search) cmd_search "$@" ;; + add) cmd_add "${1:-}" ;; + status) cmd_status ;; + rebuild) cmd_rebuild ;; + help|--help|-h) cmd_help ;; + *) + log_error "Unknown command: $command" + cmd_help + return 1 + ;; + esac +} + +main "$@" +exit $? diff --git a/.agent/scripts/memory-helper.sh b/.agent/scripts/memory-helper.sh index bfc05142a..b028e55af 100755 --- a/.agent/scripts/memory-helper.sh +++ b/.agent/scripts/memory-helper.sh @@ -41,7 +41,7 @@ readonly DEFAULT_MAX_AGE_DAYS=90 readonly STALE_WARNING_DAYS=60 # Valid learning types (matches documentation and Continuous-Claude-v3) -readonly VALID_TYPES="WORKING_SOLUTION FAILED_APPROACH CODEBASE_PATTERN USER_PREFERENCE TOOL_CONFIG DECISION CONTEXT ARCHITECTURAL_DECISION ERROR_FIX OPEN_THREAD" +readonly VALID_TYPES="WORKING_SOLUTION FAILED_APPROACH CODEBASE_PATTERN USER_PREFERENCE TOOL_CONFIG DECISION CONTEXT ARCHITECTURAL_DECISION ERROR_FIX OPEN_THREAD SUCCESS_PATTERN FAILURE_PATTERN" # Valid relation types (inspired by Supermemory's relational versioning) # - updates: New info supersedes old (state mutation) @@ -375,6 +375,7 @@ cmd_recall() { local project_filter="" local format="text" local recent_mode=false + local semantic_mode=false while [[ $# -gt 0 ]]; do case "$1" in @@ -384,6 +385,7 @@ cmd_recall() { --max-age-days) max_age_days="$2"; shift 2 ;; --project|-p) project_filter="$2"; shift 2 ;; --recent) recent_mode=true; limit="${2:-10}"; shift; [[ "${1:-}" =~ ^[0-9]+$ ]] && shift ;; + --semantic|--similar) semantic_mode=true; shift ;; --format) format="$2"; shift 2 ;; --json) format="json"; shift ;; --stats) cmd_stats; return 0 ;; @@ -418,6 +420,22 @@ cmd_recall() { return 1 fi + # Handle --semantic mode (delegate to embeddings helper) + if [[ "$semantic_mode" == true ]]; then + local embeddings_script + embeddings_script="$(dirname "$0")/memory-embeddings-helper.sh" + if [[ ! -x "$embeddings_script" ]]; then + log_error "Semantic search not available. Run: memory-embeddings-helper.sh setup" + return 1 + fi + local semantic_args=("search" "$query" "--limit" "$limit") + if [[ "$format" == "json" ]]; then + semantic_args+=("--json") + fi + "$embeddings_script" "${semantic_args[@]}" + return $? + fi + # Escape query for FTS5 - escape both single and double quotes local escaped_query="${query//\'/\'\'}" escaped_query="${escaped_query//\"/\"\"}" @@ -1054,7 +1072,8 @@ STORE OPTIONS: VALID TYPES: WORKING_SOLUTION, FAILED_APPROACH, CODEBASE_PATTERN, USER_PREFERENCE, - TOOL_CONFIG, DECISION, CONTEXT, ARCHITECTURAL_DECISION, ERROR_FIX, OPEN_THREAD + TOOL_CONFIG, DECISION, CONTEXT, ARCHITECTURAL_DECISION, ERROR_FIX, + OPEN_THREAD, SUCCESS_PATTERN, FAILURE_PATTERN RELATION TYPES (inspired by Supermemory): updates - New info supersedes old (state mutation) @@ -1071,6 +1090,8 @@ RECALL OPTIONS: --max-age-days Only recent entries --project Filter by project path --recent [n] Show n most recent entries (default: 10) + --semantic Use semantic similarity search (requires embeddings setup) + --similar Alias for --semantic --stats Show memory statistics --json Output as JSON @@ -1111,9 +1132,12 @@ EXAMPLES: # Find latest version in a chain memory-helper.sh latest mem_xxx - # Recall learnings + # Recall learnings (keyword search - default) memory-helper.sh recall --query "database search" --limit 10 + # Recall learnings (semantic similarity - opt-in, requires setup) + memory-helper.sh recall --query "how to optimize queries" --semantic + # Check for stale entries memory-helper.sh validate diff --git a/.agent/scripts/pattern-tracker-helper.sh b/.agent/scripts/pattern-tracker-helper.sh new file mode 100755 index 000000000..ba3534ac6 --- /dev/null +++ b/.agent/scripts/pattern-tracker-helper.sh @@ -0,0 +1,386 @@ +#!/usr/bin/env bash +# pattern-tracker-helper.sh - Track and analyze success/failure patterns +# Extends memory-helper.sh with pattern-specific analysis +# +# Usage: +# pattern-tracker-helper.sh record --outcome success --task-type "code-review" \ +# --model sonnet --description "Used structured review checklist" +# pattern-tracker-helper.sh record --outcome failure --task-type "refactor" \ +# --model haiku --description "Haiku missed edge cases in complex refactor" +# pattern-tracker-helper.sh analyze [--task-type TYPE] [--model MODEL] +# pattern-tracker-helper.sh suggest "task description" +# pattern-tracker-helper.sh stats +# pattern-tracker-helper.sh help + +set -euo pipefail + +# Configuration +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +readonly SCRIPT_DIR +readonly MEMORY_HELPER="$SCRIPT_DIR/memory-helper.sh" + +# Colors +readonly RED='\033[0;31m' +readonly GREEN='\033[0;32m' +readonly YELLOW='\033[0;33m' +readonly BLUE='\033[0;34m' +readonly CYAN='\033[0;36m' +readonly NC='\033[0m' + +log_info() { echo -e "${BLUE}[INFO]${NC} $*"; } +log_success() { echo -e "${GREEN}[OK]${NC} $*"; } +log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; } +log_error() { echo -e "${RED}[ERROR]${NC} $*" >&2; } + +# Valid task types for pattern tracking +readonly VALID_TASK_TYPES="code-review refactor bugfix feature docs testing deployment security architecture planning research content seo" + +# Valid model tiers +readonly VALID_MODELS="haiku flash sonnet pro opus" + +####################################### +# Record a success or failure pattern +####################################### +cmd_record() { + local outcome="" + local task_type="" + local model="" + local description="" + local tags="" + + while [[ $# -gt 0 ]]; do + case "$1" in + --outcome) outcome="$2"; shift 2 ;; + --task-type) task_type="$2"; shift 2 ;; + --model) model="$2"; shift 2 ;; + --description|--desc) description="$2"; shift 2 ;; + --tags) tags="$2"; shift 2 ;; + *) + if [[ -z "$description" ]]; then + description="$1" + fi + shift + ;; + esac + done + + # Validate required fields + if [[ -z "$outcome" ]]; then + log_error "Outcome required: --outcome success|failure" + return 1 + fi + + if [[ "$outcome" != "success" && "$outcome" != "failure" ]]; then + log_error "Outcome must be 'success' or 'failure'" + return 1 + fi + + if [[ -z "$description" ]]; then + log_error "Description required: --description \"what happened\"" + return 1 + fi + + # Validate task type if provided + if [[ -n "$task_type" ]]; then + local type_pattern=" $task_type " + if [[ ! " $VALID_TASK_TYPES " =~ $type_pattern ]]; then + log_warn "Non-standard task type: $task_type (standard: $VALID_TASK_TYPES)" + fi + fi + + # Validate model if provided + if [[ -n "$model" ]]; then + local model_pattern=" $model " + if [[ ! " $VALID_MODELS " =~ $model_pattern ]]; then + log_warn "Non-standard model: $model (standard: $VALID_MODELS)" + fi + fi + + # Build memory type + local memory_type + if [[ "$outcome" == "success" ]]; then + memory_type="SUCCESS_PATTERN" + else + memory_type="FAILURE_PATTERN" + fi + + # Build tags + local all_tags="pattern" + [[ -n "$task_type" ]] && all_tags="$all_tags,$task_type" + [[ -n "$model" ]] && all_tags="$all_tags,model:$model" + [[ -n "$tags" ]] && all_tags="$all_tags,$tags" + + # Build content with structured metadata + local content="$description" + [[ -n "$task_type" ]] && content="[task:$task_type] $content" + [[ -n "$model" ]] && content="$content [model:$model]" + + # Store via memory-helper.sh + "$MEMORY_HELPER" store \ + --content "$content" \ + --type "$memory_type" \ + --tags "$all_tags" \ + --confidence "high" + + log_success "Recorded $outcome pattern: $description" + return 0 +} + +####################################### +# Analyze patterns from memory +####################################### +cmd_analyze() { + local task_type="" + local model="" + local limit=20 + + while [[ $# -gt 0 ]]; do + case "$1" in + --task-type) task_type="$2"; shift 2 ;; + --model) model="$2"; shift 2 ;; + --limit|-l) limit="$2"; shift 2 ;; + *) shift ;; + esac + done + + echo "" + echo -e "${CYAN}=== Pattern Analysis ===${NC}" + echo "" + + # Fetch success patterns + local success_query="SUCCESS_PATTERN" + [[ -n "$task_type" ]] && success_query="$success_query task:$task_type" + [[ -n "$model" ]] && success_query="$success_query model:$model" + + echo -e "${GREEN}Success Patterns:${NC}" + local success_results + success_results=$("$MEMORY_HELPER" recall --query "$success_query" --type SUCCESS_PATTERN --limit "$limit" --json 2>/dev/null || echo "[]") + + local success_count + if command -v jq &>/dev/null; then + success_count=$(echo "$success_results" | jq 'length' 2>/dev/null || echo "0") + if [[ "$success_count" -gt 0 ]]; then + echo "$success_results" | jq -r '.[] | " + \(.content)"' 2>/dev/null + else + echo " (none recorded)" + fi + else + if [[ "$success_results" != "[]" && -n "$success_results" ]]; then + echo "$success_results" + else + echo " (none recorded)" + success_count=0 + fi + fi + + echo "" + + # Fetch failure patterns + local failure_query="FAILURE_PATTERN" + [[ -n "$task_type" ]] && failure_query="$failure_query task:$task_type" + [[ -n "$model" ]] && failure_query="$failure_query model:$model" + + echo -e "${RED}Failure Patterns:${NC}" + local failure_results + failure_results=$("$MEMORY_HELPER" recall --query "$failure_query" --type FAILURE_PATTERN --limit "$limit" --json 2>/dev/null || echo "[]") + + local failure_count + if command -v jq &>/dev/null; then + failure_count=$(echo "$failure_results" | jq 'length' 2>/dev/null || echo "0") + if [[ "$failure_count" -gt 0 ]]; then + echo "$failure_results" | jq -r '.[] | " - \(.content)"' 2>/dev/null + else + echo " (none recorded)" + fi + else + if [[ "$failure_results" != "[]" && -n "$failure_results" ]]; then + echo "$failure_results" + else + echo " (none recorded)" + failure_count=0 + fi + fi + + echo "" + + # Summary + echo -e "${CYAN}Summary:${NC}" + echo " Successes: ${success_count:-0}" + echo " Failures: ${failure_count:-0}" + if [[ -n "$task_type" ]]; then + echo " Task type: $task_type" + fi + if [[ -n "$model" ]]; then + echo " Model: $model" + fi + echo "" + return 0 +} + +####################################### +# Suggest approach based on patterns +####################################### +cmd_suggest() { + local task_desc="$*" + + if [[ -z "$task_desc" ]]; then + log_error "Task description required: pattern-tracker-helper.sh suggest \"description\"" + return 1 + fi + + echo "" + echo -e "${CYAN}=== Pattern Suggestions for: \"$task_desc\" ===${NC}" + echo "" + + # Search for relevant success patterns + echo -e "${GREEN}What has worked before:${NC}" + local success_results + success_results=$("$MEMORY_HELPER" recall --query "$task_desc" --type SUCCESS_PATTERN --limit 5 --json 2>/dev/null || echo "[]") + + if command -v jq &>/dev/null; then + local success_count + success_count=$(echo "$success_results" | jq 'length' 2>/dev/null || echo "0") + if [[ "$success_count" -gt 0 ]]; then + echo "$success_results" | jq -r '.[] | " + \(.content) (score: \(.score // "N/A"))"' 2>/dev/null + else + echo " (no matching success patterns)" + fi + else + echo " (install jq for formatted output)" + fi + + echo "" + + # Search for relevant failure patterns + echo -e "${RED}What to avoid:${NC}" + local failure_results + failure_results=$("$MEMORY_HELPER" recall --query "$task_desc" --type FAILURE_PATTERN --limit 5 --json 2>/dev/null || echo "[]") + + if command -v jq &>/dev/null; then + local failure_count + failure_count=$(echo "$failure_results" | jq 'length' 2>/dev/null || echo "0") + if [[ "$failure_count" -gt 0 ]]; then + echo "$failure_results" | jq -r '.[] | " - \(.content) (score: \(.score // "N/A"))"' 2>/dev/null + else + echo " (no matching failure patterns)" + fi + else + echo " (install jq for formatted output)" + fi + + echo "" + return 0 +} + +####################################### +# Show pattern statistics +####################################### +cmd_stats() { + echo "" + echo -e "${CYAN}=== Pattern Statistics ===${NC}" + echo "" + + # Count by type + local success_count failure_count + success_count=$("$MEMORY_HELPER" recall --query "SUCCESS_PATTERN" --type SUCCESS_PATTERN --limit 100 --json 2>/dev/null | jq 'length' 2>/dev/null || echo "0") + failure_count=$("$MEMORY_HELPER" recall --query "FAILURE_PATTERN" --type FAILURE_PATTERN --limit 100 --json 2>/dev/null | jq 'length' 2>/dev/null || echo "0") + + echo " Success patterns: $success_count" + echo " Failure patterns: $failure_count" + echo " Total patterns: $(( success_count + failure_count ))" + echo "" + + # Show task type breakdown if jq available + if command -v jq &>/dev/null; then + echo " Task types with patterns:" + for task_type in $VALID_TASK_TYPES; do + local type_count + type_count=$("$MEMORY_HELPER" recall --query "task:$task_type" --limit 100 --json 2>/dev/null | jq 'length' 2>/dev/null || echo "0") + if [[ "$type_count" -gt 0 ]]; then + echo " $task_type: $type_count" + fi + done + echo "" + fi + return 0 +} + +####################################### +# Show help +####################################### +cmd_help() { + cat <<'EOF' +pattern-tracker-helper.sh - Track and analyze success/failure patterns + +USAGE: + pattern-tracker-helper.sh [options] + +COMMANDS: + record Record a success or failure pattern + analyze Analyze patterns by task type or model + suggest Get suggestions based on past patterns for a task + stats Show pattern statistics + help Show this help + +RECORD OPTIONS: + --outcome Required: was this a success or failure? + --task-type Task category (code-review, refactor, bugfix, etc.) + --model Model used (haiku, flash, sonnet, pro, opus) + --description What happened (required) + --tags Additional comma-separated tags + +ANALYZE OPTIONS: + --task-type Filter by task type + --model Filter by model tier + --limit Max results per category (default: 20) + +VALID TASK TYPES: + code-review, refactor, bugfix, feature, docs, testing, deployment, + security, architecture, planning, research, content, seo + +EXAMPLES: + # Record a success + pattern-tracker-helper.sh record --outcome success \ + --task-type code-review --model sonnet \ + --description "Structured checklist caught 3 bugs" + + # Record a failure + pattern-tracker-helper.sh record --outcome failure \ + --task-type refactor --model haiku \ + --description "Haiku missed edge cases in complex refactor" + + # Analyze patterns for a task type + pattern-tracker-helper.sh analyze --task-type bugfix + + # Get suggestions for a new task + pattern-tracker-helper.sh suggest "refactor the auth middleware" + + # View statistics + pattern-tracker-helper.sh stats +EOF + return 0 +} + +####################################### +# Main entry point +####################################### +main() { + local command="${1:-help}" + shift || true + + case "$command" in + record) cmd_record "$@" ;; + analyze) cmd_analyze "$@" ;; + suggest) cmd_suggest "$@" ;; + stats) cmd_stats ;; + help|--help|-h) cmd_help ;; + *) + log_error "Unknown command: $command" + cmd_help + return 1 + ;; + esac +} + +main "$@" +exit $? diff --git a/.agent/subagent-index.toon b/.agent/subagent-index.toon index 11d120ba6..3a618e4af 100644 --- a/.agent/subagent-index.toon +++ b/.agent/subagent-index.toon @@ -22,8 +22,8 @@ flash,gemini-2.5-flash,Fast cheap large context pro,gemini-2.5-pro,Capable large context --> - - + +## Quick Reference + +- **Purpose**: Route tasks to the cheapest model that can handle them well +- **Philosophy**: Use the smallest model that produces acceptable quality +- **Default**: sonnet (best balance of cost/capability for most tasks) + +## Model Tiers + +| Tier | Model | Cost | Best For | +|------|-------|------|----------| +| `haiku` | claude-3-5-haiku | Lowest | Triage, classification, simple transforms, formatting | +| `flash` | gemini-2.5-flash | Low | Large context reads, summarization, bulk processing | +| `sonnet` | claude-sonnet-4 | Medium | Code implementation, review, most development tasks | +| `pro` | gemini-2.5-pro | Medium-High | Large codebase analysis, complex reasoning with big context | +| `opus` | claude-opus-4 | Highest | Architecture decisions, complex multi-step reasoning, novel problems | + +## Routing Rules + +### Use `haiku` when: + +- Classifying or triaging (bug vs feature, priority assignment) +- Simple text transforms (rename, reformat, extract fields) +- Generating commit messages from diffs +- Answering factual questions about code (no reasoning needed) +- Routing decisions (which subagent to use) + +### Use `flash` when: + +- Reading large files or codebases (>50K tokens of context) +- Summarizing documents, PRs, or discussions +- Bulk processing (many small tasks in sequence) +- Initial research sweeps before deeper analysis + +### Use `sonnet` when (default): + +- Writing or modifying code +- Code review with actionable feedback +- Debugging with reasoning +- Creating documentation from code +- Most interactive development tasks + +### Use `pro` when: + +- Analyzing very large codebases (>100K tokens) +- Complex reasoning that also needs large context +- Multi-file refactoring across many files + +### Use `opus` when: + +- Architecture and system design decisions +- Novel problem-solving (no existing patterns to follow) +- Security audits requiring deep reasoning +- Complex multi-step plans with dependencies +- Evaluating trade-offs with many variables + +## Subagent Frontmatter + +Add `model:` to subagent YAML frontmatter to declare the recommended tier: + +```yaml +--- +description: Simple text formatting utility +mode: subagent +model: haiku +tools: + read: true +--- +``` + +Valid values: `haiku`, `flash`, `sonnet`, `pro`, `opus` + +When `model:` is absent, `sonnet` is assumed (the default tier). + +## Cost Estimation + +Approximate relative costs (sonnet = 1x baseline): + +| Tier | Input Cost | Output Cost | Relative | +|------|-----------|-------------|----------| +| haiku | 0.25x | 0.25x | ~0.25x | +| flash | 0.15x | 0.30x | ~0.20x | +| sonnet | 1x | 1x | 1x | +| pro | 1.25x | 2.5x | ~1.5x | +| opus | 3x | 3x | ~3x | + +## Integration with Task Tool + +When using the Task tool to dispatch subagents, the `model:` field in the subagent's frontmatter serves as a recommendation. The orchestrating agent can override based on task complexity. + + + +## Decision Flowchart + +```text +Is the task simple classification/formatting? + → YES: haiku + → NO: Does it need >50K tokens of context? + → YES: Is deep reasoning also needed? + → YES: pro + → NO: flash + → NO: Is it a novel architecture/design problem? + → YES: opus + → NO: sonnet +``` + +## Examples + +| Task | Recommended | Why | +|------|-------------|-----| +| "Rename variable X to Y across files" | haiku | Simple text transform | +| "Summarize this 200-page PDF" | flash | Large context, low reasoning | +| "Fix this React component bug" | sonnet | Code + reasoning | +| "Review this 500-file PR" | pro | Large context + reasoning | +| "Design the auth system architecture" | opus | Novel design, trade-offs | +| "Generate a commit message" | haiku | Simple text generation | +| "Write unit tests for this module" | sonnet | Code generation | +| "Evaluate 3 database options for our use case" | opus | Complex trade-off analysis | diff --git a/README.md b/README.md index 4f1862a4a..c46396613 100644 --- a/README.md +++ b/README.md @@ -92,9 +92,9 @@ The result: AI agents that work *with* your development process, not around it. ### Agent Structure - Primary agents (Build+, SEO, Marketing, etc.) with @plan-plus subagent for planning-only mode -- 572+ subagent markdown files organized by domain -- 157 helper scripts in `.agent/scripts/` -- 22 slash commands for common workflows +- 614+ subagent markdown files organized by domain +- 163 helper scripts in `.agent/scripts/` +- 28 slash commands for common workflows @@ -480,7 +480,7 @@ aidevops implements proven agent design patterns identified by [Lance Martin (La | Pattern | Description | aidevops Implementation | |---------|-------------|------------------------| -| **Give Agents a Computer** | Filesystem + shell for persistent context | `~/.aidevops/.agent-workspace/`, 157 helper scripts | +| **Give Agents a Computer** | Filesystem + shell for persistent context | `~/.aidevops/.agent-workspace/`, 163 helper scripts | | **Multi-Layer Action Space** | Few tools, push actions to computer | Per-agent MCP filtering (~12-20 tools each) | | **Progressive Disclosure** | Load context on-demand | Subagent routing with content summaries, YAML frontmatter, read-on-demand | | **Offload Context** | Write results to filesystem | `.agent-workspace/work/[project]/` for persistence | @@ -1252,7 +1252,7 @@ aidevops is registered as a **Claude Code plugin marketplace**. Install with two /plugin install aidevops@aidevops ``` -This installs the complete framework: 14 primary agents, 572+ subagents, and 157 helper scripts. +This installs the complete framework: 14 primary agents, 614+ subagents, and 163 helper scripts. ### Importing External Skills @@ -1340,7 +1340,7 @@ Ordered as they appear in OpenCode Tab selector and other AI assistants (15 tota ### **Example Subagents with MCP Integration** -These are examples of subagents that have supporting MCPs enabled. See `.agent/` for the full list of 572+ subagents organized by domain. +These are examples of subagents that have supporting MCPs enabled. See `.agent/` for the full list of 614+ subagents organized by domain. | Agent | Purpose | MCPs Enabled | |-------|---------|--------------|