From 2b814d9556ecc57cfc94aa652eb0c6043f3f984a Mon Sep 17 00:00:00 2001 From: "Toast (gastown)" Date: Thu, 2 Apr 2026 09:49:04 +0000 Subject: [PATCH] De-duplicate README.md - remove overlapping second section - Removed entire duplicate second section (lines 708-1166, 459 lines) - Ported unique content from second section into appropriate locations: - Programmatic API examples (TokenAnalyzer/MethodAnalyzer) - Method-level filtering with pattern syntax table - Use Cases & Examples (5 practical scenarios) - Requirements, Contributing, Support sections - Created by attribution - Trimmed internal duplication (Include vs Exclude, LLM Context examples, Configuration) - Reduced from 1166 lines to 600 lines (48% reduction) - Buy Me A Coffee section and QR code preserved at top --- README.md | 738 +++++++----------------------------------------------- 1 file changed, 86 insertions(+), 652 deletions(-) diff --git a/README.md b/README.md index 850ad15..d29a519 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ If you find this tool helpful, consider buying me a coffee! Your support helps m ## Features ### ๐Ÿš€ Platform Features (v3.0.0) + - ๐Ÿ”Œ **Plugin Architecture** - Modular, extensible system for languages and exporters - ๐Ÿ”€ **Git Integration** - Analyze only changed files, diff analysis, author tracking - ๐Ÿ‘๏ธ **Watch Mode** - Real-time file monitoring and auto-analysis @@ -37,23 +38,27 @@ If you find this tool helpful, consider buying me a coffee! Your support helps m - ๐Ÿ—๏ธ **Modular Core** - Scanner, Analyzer, ContextBuilder, Reporter ### ๐ŸŽจ User Interface + - ๐Ÿง™ **Interactive Wizard Mode** - User-friendly guided setup (default) - ๐Ÿ’ป **CLI Mode** - Traditional command-line interface (--cli flag) - ๐Ÿ“ค **Interactive export** - Prompts for export choice when no options specified ### ๐Ÿ”ข Token Analysis + - โœ… **Exact token counting** using tiktoken (GPT-4 compatible) - ๐ŸŒ **Multi-language support** - 14+ languages: JavaScript, TypeScript, Python, PHP, Ruby, Java, Kotlin, C#, Go, Rust, Swift, C/C++, Scala - ๐ŸŽฏ **Method-level analysis** - Analyze tokens per function/method - ๐Ÿ“Š **Detailed reporting** - by file type, largest files, statistics ### ๐ŸŽฏ Filtering & Configuration + - ๐Ÿšซ **Dual ignore system** - respects both `.gitignore` and context ignore rules - ๐Ÿ“‹ **Include/Exclude modes** - `.contextinclude` takes priority over `.contextignore` - ๐Ÿ” **Method filtering** - `.methodinclude` and `.methodignore` for granular control - ๐ŸŽฏ **Core application focus** - configured to analyze only essential code files ### ๐Ÿ“ค Export Options + - ๐Ÿค– **LLM context export** - generate optimized file lists for LLM consumption - ๐Ÿ“‹ **Clipboard integration** - copy context directly to clipboard - ๐Ÿ’พ **JSON/YAML/CSV/XML exports** - multiple format options @@ -64,12 +69,14 @@ If you find this tool helpful, consider buying me a coffee! Your support helps m ## Quick Start ### ๐Ÿง™ Interactive Wizard Mode (Default) + ```bash # Launch interactive wizard (guides you through options) context-manager ``` The wizard provides a user-friendly interface to: + - Select your use case (Bug Fix, Feature, Code Review, etc.) - Choose target LLM (Claude, GPT-4, Gemini, etc.) - Pick output format (TOON, JSON, YAML, etc.) @@ -77,6 +84,7 @@ The wizard provides a user-friendly interface to: **Note:** Wizard mode uses Ink terminal UI. If you experience visual artifacts, use CLI mode with `--cli` flag. ### ๐Ÿ’ป CLI Mode + ```bash # Use CLI mode instead of wizard context-manager --cli @@ -97,6 +105,7 @@ context-manager --cli -g -s # GitIngest digest + detailed report ``` ### ๐Ÿค– LLM Optimization (v2.3.7) + ```bash # Auto-detect LLM from environment export ANTHROPIC_API_KEY=sk-... @@ -121,6 +130,7 @@ context-manager --cli --target-model claude-sonnet-4.5 ``` Supported LLM models (9+ models): + - **Anthropic**: Claude Sonnet 4.5, Claude Opus 4 - **OpenAI**: GPT-4 Turbo, GPT-4o, GPT-4o Mini - **Google**: Gemini 1.5 Pro, Gemini 2.0 Flash @@ -129,6 +139,7 @@ Supported LLM models (9+ models): Custom models supported via `.context-manager/custom-profiles.json` ### ๐Ÿ”€ Git Integration (v3.0.0) + ```bash # Analyze only uncommitted changes context-manager --changed-only @@ -150,6 +161,7 @@ context-manager --changed-only --with-authors ``` ### ๐Ÿ‘๏ธ Watch Mode (v3.0.0) + ```bash # Start watch mode context-manager watch @@ -168,6 +180,7 @@ context-manager watch --debounce 2000 ``` ### ๐ŸŒ API Server (v3.0.0) + ```bash # Start API server context-manager serve @@ -189,14 +202,6 @@ curl http://localhost:3000/api/v1/methods?file=src/server.js curl http://localhost:3000/api/v1/diff?since=main ``` -### Wrapper Script Usage -```bash -# Using the NPM package globally -context-manager -context-manager --save-report -context-manager --context-clipboard -``` - ## ๐Ÿงช Testing & Validation ### Test Repositories @@ -224,6 +229,7 @@ See [test-repos/README.md](test-repos/README.md) for complete testing guide. Complete manual testing guide available at [docs/MANUAL-TESTING-v3.0.md](docs/MANUAL-TESTING-v3.0.md) Includes: + - โœ… 50+ test scenarios for all v3.0.0 features - โœ… API endpoint validation - โœ… Git integration testing @@ -244,145 +250,63 @@ npm run test:api npm run test:watch ``` -## Current Configuration - -The tool is configured to focus on **core application logic only**: - -### โœ… Included (64 JS files, ~181k tokens) -- Core MCP server implementation (`utility-mcp/src/`) -- Authentication and security layers -- Request handlers and routing -- Transport protocols and communication -- Utilities and validation logic -- Configuration management -- Error handling and monitoring - -### ๐Ÿšซ Excluded via context ignore rules -- Documentation files (`.md`, `.txt`) -- Configuration files (`.json`, `.yml`) -- Infrastructure and deployment files -- Testing and script directories -- Build artifacts and dependencies -- Workflow orchestration files (`utility-mcp/src/workflows/**`) -- Testing utilities (`utility-mcp/src/testing/**`) -- All non-essential supporting files - ## Usage -### Basic Analysis ```bash -# Interactive analysis with export selection -context-manager - -# Quiet mode (no file listing) -context-manager --no-verbose - -# With detailed JSON report -context-manager --save-report - -# Generate LLM context file list -context-manager --context-export - -# Copy context directly to clipboard -context-manager --context-clipboard +context-manager # Interactive wizard (default) +context-manager --cli # CLI mode +context-manager --cli -s # Save detailed JSON report +context-manager --cli --context-clipboard # Copy context to clipboard +context-manager --cli -m # Method-level analysis +context-manager --cli -g # GitIngest digest ``` -### Interactive Export Selection - -When you run the tool without specifying export options (`--save-report`, `--context-export`, or `--context-clipboard`), it will automatically prompt you to choose an export option after the analysis: - -```bash -# Run analysis and get prompted for export options -context-manager - -# The tool will show: -# ๐Ÿ“ค Export Options: -# 1) Save detailed JSON report (token-analysis-report.json) -# 2) Generate LLM context file (llm-context.json) -# 3) Copy LLM context to clipboard -# 4) No export (skip) -# -# ๐Ÿค” Which export option would you like? (1-4): -``` - -This interactive mode ensures you never miss the opportunity to export your analysis results in the format you need. +When no export options are specified, the tool prompts for export selection interactively. ## Include vs Exclude Modes -The token calculator supports two complementary filtering modes: - -### EXCLUDE Mode (.contextignore) -- **Default mode** when only `.contextignore` exists -- Includes all files **except** those matching ignore patterns -- Traditional gitignore-style exclusion logic - -### INCLUDE Mode (.contextinclude) -- **Priority mode** - when `.contextinclude` exists, `.contextignore` is ignored -- Includes **only** files matching include patterns -- More precise control for specific file selection -- Perfect for creating focused analysis sets - -### Mode Priority -1. If `.contextinclude` exists โ†’ **INCLUDE mode** (ignore `.contextignore`) -2. If only `.contextignore` exists โ†’ **EXCLUDE mode** -3. If neither exists โ†’ Include all files (respect `.gitignore` only) - -### Example Usage -```bash -# EXCLUDE mode: Include everything except patterns in .contextignore -rm .contextinclude # Remove include file -context-manager - -# INCLUDE mode: Include only patterns in .contextinclude -# (automatically ignores .contextignore) -context-manager -``` - -### Help and Options -```bash -context-manager --help -``` +When `.contextinclude` exists, it activates **INCLUDE mode** (ignoring `.contextignore`). Otherwise, **EXCLUDE mode** uses `.contextignore` patterns. See the Configuration section below for details and examples. ### Available Options + - `--save-report`, `-s` - Save detailed JSON report - `--no-verbose` - Disable file listing (verbose is default) - `--context-export` - Generate LLM context file list (saves as llm-context.json) - `--context-clipboard` - Copy LLM context directly to clipboard - `--detailed-context` - Use detailed context format (8.6k chars, default is compact 1.2k) +- `--method-level`, `-m` - Enable method-level analysis - `--help`, `-h` - Show help message ## LLM Context Export -The token calculator can generate optimized file lists for LLM consumption, with two format options: +The token calculator generates optimized file lists for LLM consumption, with two format options: ### Ultra-Compact Format (Default) + - **Size**: ~2.3k characters (structured JSON) - **Content**: Project metadata and organized file paths without token counts -- **Format**: Identical to llm-context.json file - complete JSON structure -- **Perfect for**: LLM consumption, programmatic processing, structured data needs +- **Perfect for**: LLM consumption, programmatic processing - **Usage**: `--context-clipboard` or `--context-export` ### Detailed Format (Legacy) + - **Size**: ~8.6k characters (comprehensive) - **Content**: Full paths, categories, importance scores, directory stats - **Perfect for**: Initial project analysis, comprehensive documentation - **Usage**: `--detailed-context --context-clipboard` ### Features + - **Smart file selection** - Top files by token count and importance - **Directory grouping** - Common prefix compression saves space - **Token abbreviation** - "12k" instead of "12,388 tokens" -- **Extension removal** - ".js" removed to save characters - **Cross-platform clipboard** - Works on macOS, Linux, and Windows -- **Multiple output formats** - JSON file or clipboard ready text - -### Usage ```bash -# Generate minimal LLM context and save to llm-context.json (2.3k chars JSON) +# Generate minimal LLM context and save to llm-context.json context-manager --context-export -# Copy minimal context directly to clipboard (2.3k chars JSON - identical to file) +# Copy minimal context directly to clipboard context-manager --context-clipboard # Copy detailed context to clipboard (8.6k chars) @@ -392,459 +316,88 @@ context-manager --detailed-context --context-clipboard context-manager --save-report --context-clipboard ``` -### Output Format Examples - -**Compact Format (JSON - 2.3k chars):** -```json -{ - "project": { - "root": "cloudstack-go-mcp-proxy", - "totalFiles": 64, - "totalTokens": 181480 - }, - "paths": { - "utility-mcp/src/server/": [ - "CloudStackUtilityMCP.js" - ], - "utility-mcp/src/handlers/": [ - "workflow-handlers.js", - "tool-handlers.js", - "analytics-handler.js" - ], - "utility-mcp/src/utils/": [ - "security.js", - "usage-tracker.js", - "cache-warming.js" - ] - } -} -``` - -**Detailed Format (8.6k chars):** -``` -# cloudstack-go-mcp-proxy Codebase Context - -**Project:** 64 files, 181,480 tokens - -**Core Files (Top 20):** -1. `utility-mcp/src/server/CloudStackUtilityMCP.js` (12,388 tokens, server) -2. `utility-mcp/src/handlers/workflow-handlers.js` (11,007 tokens, handler) -... - -**All Files:** -```json -[{"path": "file.js", "t": 1234, "c": "core", "i": 85}] -``` - -**Use Cases** - -**Compact Format (2.3k chars JSON):** -1. **LLM Integration** - Structured data for AI assistants with complete project context -2. **Programmatic Processing** - JSON format for automated tools and scripts -3. **Context Sharing** - Identical format in clipboard and file exports -4. **Development Workflows** - Consistent structure for CI/CD and automation - -**Detailed Format (8.6k chars):** -1. **Architecture Planning** - Comprehensive project overview for major decisions -2. **New Team Member Onboarding** - Complete codebase understanding -3. **Documentation Generation** - Full project structure analysis -4. **Code Review Preparation** - Detailed file relationships and importance - -**General Use Cases:** -- Development workflow integration -- CI/CD pipeline context generation -- Automated documentation updates -- Project health monitoring - ## GitIngest Format Export -Context-manager now supports generating GitIngest-style digest files - a single, prompt-friendly text file perfect for LLM consumption. - -### What is GitIngest Format? - -GitIngest format consolidates your entire codebase into a single text file with: -- Project summary and statistics -- Visual directory tree structure -- Complete file contents with clear separators -- Token count estimates - -This format is inspired by [GitIngest](https://github.com/coderamp-labs/gitingest), implemented purely in JavaScript with zero additional dependencies. - -### Usage +Generate a single, prompt-friendly text file with your entire codebase for LLM consumption. Inspired by [GitIngest](https://github.com/coderamp-labs/gitingest). ```bash -# Standard workflow - analyze and generate digest in one step -context-manager --gitingest -context-manager -g - -# Combine with other exports -context-manager -g -s # digest.txt + token-analysis-report.json - -# Two-step workflow - generate digest from existing JSON (fast, no re-scan) -context-manager -s # Step 1: Create report -context-manager --gitingest-from-report # Step 2: Generate digest - -# Or from LLM context -context-manager --context-export # Step 1: Create context -context-manager --gitingest-from-context # Step 2: Generate digest - -# With custom filenames -context-manager --gitingest-from-report my-report.json -context-manager --gitingest-from-context my-context.json -``` - -**Why use JSON-based digest?** -- โšก **Performance**: Instant digest generation without re-scanning -- ๐Ÿ”„ **Reusability**: Generate multiple digests from one analysis -- ๐Ÿ“ฆ **Workflow**: Separate analysis from export steps -- ๐ŸŽฏ **Flexibility**: Use different JSON sources for different purposes - -### Output Example - -The generated `digest.txt` file looks like: - +context-manager --gitingest # Generate digest.txt +context-manager -g -s # digest.txt + token-analysis-report.json +context-manager --gitingest-from-report # Generate from existing JSON (no re-scan) +context-manager --gitingest-from-context # Generate from LLM context file ``` -Directory: my-project -Files analyzed: 42 - -Estimated tokens: 15.2k -Directory structure: -โ””โ”€โ”€ my-project/ - โ”œโ”€โ”€ src/ - โ”‚ โ”œโ”€โ”€ index.js - โ”‚ โ””โ”€โ”€ utils.js - โ””โ”€โ”€ README.md - - -================================================ -FILE: src/index.js -================================================ -[complete file contents here] - -================================================ -FILE: src/utils.js -================================================ -[complete file contents here] -``` - -### Key Features - -- **Single File**: Everything in one file for easy LLM ingestion -- **Tree Visualization**: Clear directory structure -- **Token Estimates**: Formatted as "1.2k" or "1.5M" -- **Sorted Output**: Files sorted by token count (largest first) -- **Filter Compatible**: Respects all `.gitignore` and context ignore rules - -### Use Cases -1. **LLM Context Windows**: Paste entire codebase as single context -2. **Code Reviews**: Share complete project snapshot -3. **Documentation**: Single-file project reference -4. **AI Analysis**: Perfect for ChatGPT, Claude, or other LLMs -5. **Archival**: Simple project snapshot format - -### Version Tracking - -Context-manager implements GitIngest format v0.3.1. See [docs/GITINGEST_VERSION.md](docs/GITINGEST_VERSION.md) for implementation details and version history. +Output includes project summary, directory tree, and complete file contents sorted by token count. See [docs/GITINGEST_VERSION.md](docs/GITINGEST_VERSION.md) for version details. ## Configuration -### .contextignore File (EXCLUDE Mode) - -The `.contextignore` file is pre-configured for core application analysis: +### File-Level Filtering -```bash -# Current focus: Only core JS files in utility-mcp/src/ -# Excludes: -**/*.md # All documentation -**/*.json # All configuration files -**/*.yml # All YAML files -infrastructure/** # Infrastructure code -workflows/** # Workflow definitions -docs/** # Documentation directory -token-analysis/** # Analysis tools themselves -utility-mcp/scripts/** # Utility scripts -utility-mcp/src/workflows/** # Workflow JS files -utility-mcp/src/testing/** # Testing utilities -``` +**Priority Order:** -### .contextinclude File (INCLUDE Mode) +1. `.gitignore` (always respected) +2. `.contextinclude` - INCLUDE mode (highest priority for files) +3. `.contextignore` - EXCLUDE mode (fallback for files) -The `.contextinclude` file provides precise file selection: +**`.contextinclude`** (INCLUDE mode) - include only matching files: ```bash -# Include only core JavaScript files -# This should produce exactly 64 files - -# Include main entry point -utility-mcp/index.js - -# Include all src JavaScript files EXCEPT workflows and testing utility-mcp/src/**/*.js - -# Exclude specific subdirectories (using negation) !utility-mcp/src/workflows/** !utility-mcp/src/testing/** ``` -### Creating Custom Configurations - -**For EXCLUDE mode** (edit `.contextignore`): -```bash -# Remove lines to include more file types -# Add patterns to exclude specific files - -# Example: Include documentation -# **/*.md <- comment out or remove this line - -# Example: Exclude specific large files -your-large-file.js -specific-directory/** -``` +**`.contextignore`** (EXCLUDE mode) - exclude matching files: -**For INCLUDE mode** (create `.contextinclude`): ```bash -# Include specific files or patterns -src/**/*.js # All JS files in src -config/*.json # Config files only -docs/api/**/*.md # API documentation only - -# Use negation to exclude from broad patterns -src/**/*.js -!src/legacy/** # Exclude legacy code -!src/**/*.test.js # Exclude test files -``` - -## Configuration File Priority - -1. **`.gitignore`** (project root) - Standard git exclusions (always respected) -2. **`.contextinclude`** (token-analysis/) - INCLUDE mode (highest priority) -3. **`.contextignore`** (token-analysis/) - EXCLUDE mode (used when no include file) -4. **`.contextignore`** (project root) - Fallback EXCLUDE mode location - -## Installation - -For exact token counting, install tiktoken: - -```bash -npm install tiktoken +**/*.md +**/*.json +**/*.yml +node_modules/** +**/*.test.js ``` -Without tiktoken, the tool uses smart estimation (~95% accuracy). +Use negation (`!pattern`) to exclude from broad include patterns. ## Output Example ``` ๐ŸŽฏ PROJECT TOKEN ANALYSIS REPORT -================================================================================ ๐Ÿ“Š Total files analyzed: 64 ๐Ÿ”ข Total tokens: 181,480 ๐Ÿ’พ Total size: 0.78 MB ๐Ÿ“„ Total lines: 28,721 -๐Ÿ“ˆ Average tokens per file: 2,836 -๐Ÿšซ Files ignored by .gitignore: 11,912 -๐Ÿ“‹ Files ignored by calculator rules: 198 - -๐Ÿ“‹ BY FILE TYPE: --------------------------------------------------------------------------------- -Extension Files Tokens Size (KB) Lines --------------------------------------------------------------------------------- -.js 64 181,480 799.8 28,721 - -๐Ÿ† TOP 5 LARGEST FILES BY TOKEN COUNT: --------------------------------------------------------------------------------- - 1. 12,388 tokens (6.8%) - utility-mcp/src/server/CloudStackUtilityMCP.js - 2. 11,007 tokens (6.1%) - utility-mcp/src/handlers/workflow-handlers.js - 3. 7,814 tokens (4.3%) - utility-mcp/src/utils/security.js - 4. 6,669 tokens (3.7%) - utility-mcp/src/handlers/tool-handlers.js - 5. 5,640 tokens (3.1%) - utility-mcp/src/ci-cd/pipeline-integration.js -``` - -## Context Management -Perfect for LLM context window optimization: -- **181k tokens** = Core application logic only -- **Clean analysis** = No noise from docs, configs, or build files -- **Focused development** = Essential code for AI-assisted development -- **Context efficiency** = Maximum useful code per token -- **Dual mode flexibility** = Precise include/exclude control -- **Ultra-minimal export** = 1k chars (89% reduction) for frequent AI interactions -- **Detailed export** = 8.6k chars for comprehensive analysis when needed - -## Integration - -You can integrate this tool into: -- CI/CD pipelines for code size monitoring -- Pre-commit hooks for token budget checks -- Documentation generation workflows -- Code quality gates -- LLM context preparation workflows -- Development environment setup +๐Ÿ† TOP 5 LARGEST FILES: + 1. 12,388 tokens - utility-mcp/src/server/CloudStackUtilityMCP.js + 2. 11,007 tokens - utility-mcp/src/handlers/workflow-handlers.js + 3. 7,814 tokens - utility-mcp/src/utils/security.js +``` ## Troubleshooting ### Include vs Exclude Mode Issues -- **INCLUDE mode active**: Remove `.contextinclude` to use EXCLUDE mode -- **Wrong files included**: Check if `.contextinclude` exists (takes priority) -- **Mode confusion**: Use verbose mode to see which mode is active + +- Remove `.contextinclude` to use EXCLUDE mode +- Check if `.contextinclude` exists (takes priority) +- Use verbose mode to see which mode is active ### Patterns Not Working + - Ensure no inline comments in ignore/include pattern files - Use file patterns (`docs/**`) instead of directory patterns (`docs/`) -- Test specific patterns with verbose mode - Check pattern syntax: `**` for recursive, `*` for single level ### Token Count Issues + - **Too high**: Review included files with verbose mode, add exclusion patterns - **Too low**: Check if important files are excluded, review patterns - **Inconsistent**: Verify which mode is active (include vs exclude) -### Missing Expected Files -- Check if files are excluded by `.gitignore` (always respected) -- Verify calculator ignore/include patterns -- Ensure files are recognized as text files -- Use verbose mode to see exclusion reasons - -# Context Manager - -LLM context manager with method-level filtering and token optimization. The ultimate tool for AI-assisted development. - -*Created by Hakkฤฑ SaฤŸdฤฑรง* - -## ๐Ÿš€ Features - -โœ… **File-level token analysis** - Analyze entire files and directories -๐Ÿ”ง **Method-level analysis** - Extract and analyze specific methods from JavaScript/TypeScript/Rust/C#/Go/Java -๐Ÿ“‹ **Dual filtering system** - Include/exclude files and methods with pattern matching -๐Ÿ“Š **LLM context optimization** - Generate ultra-compact context for AI assistants -๐ŸŽฏ **Exact token counting** - Uses tiktoken for GPT-4 compatible counts -๐Ÿ“ค **Multiple export formats** - JSON reports, clipboard, file exports -๐Ÿ“ฆ **NPM package** - Use programmatically or as global CLI tool -๐Ÿ” **Pattern matching** - Wildcards and regex support for flexible filtering -โšก **Performance optimized** - 36% smaller codebase with enhanced functionality - -## ๐Ÿ“ฆ Installation - -### Option 1: NPM Package (Recommended) -```bash -# Local installation -npm install @hakkisagdic/context-manager - -# Global installation -npm install -g @hakkisagdic/context-manager - -# Run globally -context-manager --help -``` - -### Option 2: Direct Usage -```bash -# Clone and use directly -git clone -cd token-analysis -node token-calculator.js --help -``` - -## ๐ŸŽฏ Quick Start - -### Basic Analysis -```bash -# Interactive analysis with export selection -context-manager - -# File-level analysis with clipboard export -context-manager --context-clipboard - -# Method-level analysis -context-manager --method-level --context-export - -# Analysis with reports -context-manager --method-level --save-report --verbose -``` - -### Advanced Usage -```bash -# Focus on specific methods only -echo "calculateTokens\nhandleRequest\n*Validator" > .methodinclude -context-manager --method-level - -# Exclude test methods -echo "*test*\n*debug*\nconsole" > .methodignore -context-manager --method-level --context-clipboard -``` - -## Usage - -### Command Line Interface - -```bash -# Basic analysis -context-manager - -# Method-level analysis -context-manager --method-level - -# Save detailed report -context-manager --save-report - -# Copy context to clipboard -context-manager --context-clipboard - -# Combine options -context-manager --method-level --save-report --verbose -``` - -### Programmatic Usage - -```javascript -const { TokenAnalyzer } = require('@hakkisagdic/context-manager'); - -// Basic file-level analysis -const analyzer = new TokenAnalyzer('./src', { - methodLevel: false, - verbose: true -}); - -// Method-level analysis -const methodAnalyzer = new TokenAnalyzer('./src', { - methodLevel: true, - saveReport: true -}); - -analyzer.run(); -``` - -## ๐Ÿ”ง Configuration - -### File-Level Filtering - -**Priority Order:** -1. `.gitignore` (project root) - Standard git exclusions (always respected) -2. `.contextinclude` - INCLUDE mode (highest priority for files) -3. `.contextignore` - EXCLUDE mode (fallback for files) - -**`.contextinclude`** - Include only these files: -```bash -# Include only core JavaScript files -utility-mcp/src/**/*.js -!utility-mcp/src/testing/** -!utility-mcp/src/workflows/** -``` - -**`.contextignore`** - Exclude these files: -```bash -# Exclude documentation and config -**/*.md -**/*.json -node_modules/** -test/ -**/*.test.js -**/*.spec.js -``` - -### Method-Level Filtering +### Advanced Method Filtering **`.methodinclude`** - Include only these methods: + ```bash # Core business logic methods calculateTokens @@ -862,6 +415,7 @@ TokenCalculator.* # All methods in TokenCalculator class ``` **`.methodignore`** - Exclude these methods: + ```bash # Utility and debug methods console @@ -876,98 +430,19 @@ server.printStatus utils.debugLog ``` -### Pattern Syntax - -| Pattern | Description | Example | -|---------|-------------|----------| -| `methodName` | Exact match | `calculateTokens` | -| `*pattern*` | Contains pattern | `*Handler` matches `requestHandler` | -| `Class.*` | All methods in class | `TokenCalculator.*` | -| `file.method` | Specific file method | `server.handleRequest` | -| `!pattern` | Negation (exclude) | `!*test*` | - -## ๐Ÿ“ค Output Formats - -### 1. File-Level Context (Default) -**Use case:** General codebase analysis, file organization - -```json -{ - "project": { - "root": "my-project", - "totalFiles": 64, - "totalTokens": 181480 - }, - "paths": { - "src/core/": ["server.js", "handler.js"], - "src/utils/": ["helper.js", "validator.js"] - } -} -``` - -### 2. Method-Level Context (`--method-level`) -**Use case:** Focused analysis, debugging specific methods, LLM context optimization - -```json -{ - "project": { - "root": "my-project", - "totalFiles": 64, - "totalTokens": 181480 - }, - "methods": { - "src/server.js": [ - {"name": "handleRequest", "line": 15, "tokens": 234}, - {"name": "validateInput", "line": 45, "tokens": 156} - ], - "src/utils.js": [ - {"name": "processData", "line": 12, "tokens": 89} - ] - }, - "methodStats": { - "totalMethods": 150, - "includedMethods": 23, - "totalMethodTokens": 5670 - } -} -``` - -### 3. Detailed Report (JSON) -**Use case:** Comprehensive analysis, CI/CD integration, historical tracking - -```json -{ - "metadata": { - "generatedAt": "2024-01-15T10:30:00.000Z", - "projectRoot": "/path/to/project", - "gitignoreRules": ["node_modules/**", "*.log"], - "calculatorRules": ["src/**/*.js", "!src/test/**"] - }, - "summary": { - "totalFiles": 64, - "totalTokens": 181480, - "byExtension": {".js": {"count": 64, "tokens": 181480}}, - "largestFiles": [...] - }, - "files": [...] -} -``` - -## CLI Options +#### Method Pattern Syntax -| Option | Short | Description | -|--------|-------|-------------| -| `--save-report` | `-s` | Save detailed JSON report | -| `--verbose` | `-v` | Show included files and directories | -| `--context-export` | | Generate LLM context file | -| `--context-clipboard` | | Copy context to clipboard | -| `--method-level` | `-m` | Enable method-level analysis | -| `--help` | `-h` | Show help message | +| Pattern | Description | Example | +| ------------- | -------------------- | ----------------------------------- | +| `methodName` | Exact match | `calculateTokens` | +| `*pattern*` | Contains pattern | `*Handler` matches `requestHandler` | +| `Class.*` | All methods in class | `TokenCalculator.*` | +| `file.method` | Specific file method | `server.handleRequest` | +| `!pattern` | Negation (exclude) | `!*test*` | ## ๐Ÿ“Š Use Cases & Examples -### 1. ๐Ÿค– LLM Context Optimization -**Goal:** Generate minimal context for AI assistants +### 1. LLM Context Optimization ```bash # Ultra-compact method-level context @@ -978,10 +453,7 @@ echo "handleRequest\nprocessData\nvalidateInput" > .methodinclude context-manager --method-level --context-export ``` -**Result:** 89% smaller context compared to full codebase - -### 2. ๐Ÿ“Š Codebase Analysis -**Goal:** Understand project complexity and structure +### 2. Codebase Analysis ```bash # Analysis with detailed reports @@ -991,8 +463,7 @@ context-manager --save-report --verbose context-manager --method-level --save-report ``` -### 3. ๐Ÿ” Method-Level Debugging -**Goal:** Focus on specific problematic methods +### 3. Method-Level Debugging ```bash # Debug authentication methods only @@ -1004,8 +475,7 @@ echo "*test*\n*debug*\nconsole\nlogger" > .methodignore context-manager --method-level ``` -### 4. ๐Ÿš€ CI/CD Integration -**Goal:** Monitor codebase growth and complexity +### 4. CI/CD Integration ```bash # Daily token analysis for monitoring @@ -1015,8 +485,7 @@ context-manager --save-report > reports/analysis-$(date +%Y%m%d).json context-manager --method-level --save-report ``` -### 5. ๐Ÿ“ˆ Code Quality Gates -**Goal:** Ensure code stays within token budgets +### 5. Code Quality Gates ```bash # Check if codebase exceeds LLM context limits @@ -1027,38 +496,6 @@ if [ $TOKENS -gt 100000 ]; then fi ``` -## ๐Ÿ› ๏ธ CLI Reference - -### Core Options - -| Option | Short | Description | Example | -|--------|-------|-------------|----------| -| `--save-report` | `-s` | Save detailed JSON report | `context-manager -s` | -| `--verbose` | `-v` | Show included files/methods | `context-manager -v` | -| `--context-export` | | Generate LLM context file | `context-manager --context-export` | -| `--context-clipboard` | | Copy context to clipboard | `context-manager --context-clipboard` | -| `--method-level` | `-m` | Enable method-level analysis | `context-manager -m` | -| `--help` | `-h` | Show help message | `context-manager -h` | - -### Usage Patterns - -```bash -# Quick analysis with interactive export -context-manager - -# Method-level analysis with all outputs -context-manager --method-level --save-report --context-export --verbose - -# LLM-optimized context generation -context-manager --method-level --context-clipboard - -# CI/CD monitoring -context-manager --save-report --context-export - -# Development debugging -context-manager --method-level --verbose -``` - ## ๐Ÿ’ป Programmatic API ### Basic Usage @@ -1069,7 +506,7 @@ const { TokenAnalyzer } = require('@hakkisagdic/context-manager'); // File-level analysis const analyzer = new TokenAnalyzer('./src', { verbose: true, - saveReport: true + saveReport: true, }); analyzer.run(); @@ -1084,7 +521,7 @@ const { TokenAnalyzer, MethodAnalyzer } = require('@hakkisagdic/context-manager' const analyzer = new TokenAnalyzer('./src', { methodLevel: true, contextExport: true, - verbose: false + verbose: false, }); analyzer.run(); @@ -1100,17 +537,17 @@ const methods = methodAnalyzer.extractMethods(fileContent, 'server.js'); const analyzer = new TokenAnalyzer('./src', { // Enable method-level analysis methodLevel: true, - + // Output options saveReport: true, contextExport: true, contextToClipboard: true, - + // Verbosity verbose: true, - + // Compact context (for LLM optimization) - compactContext: true + compactContext: true, }); // Access results @@ -1124,17 +561,14 @@ console.log('Analysis complete!'); const { MethodAnalyzer, MethodFilterParser } = require('@hakkisagdic/context-manager'); // Create custom method filter -const filter = new MethodFilterParser( - './custom-methods.include', - './custom-methods.ignore' -); +const filter = new MethodFilterParser('./custom-methods.include', './custom-methods.ignore'); // Analyze specific file const methodAnalyzer = new MethodAnalyzer(); const methods = methodAnalyzer.extractMethods(content, filePath); // Filter methods -const filteredMethods = methods.filter(method => +const filteredMethods = methods.filter((method) => filter.shouldIncludeMethod(method.name, fileName) ); ``` @@ -1163,4 +597,4 @@ MIT License - see LICENSE file for details --- -*Created with โค๏ธ by Hakkฤฑ SaฤŸdฤฑรง* +_Created with โค๏ธ by Hakkฤฑ SaฤŸdฤฑรง_