feat(ai): auto-evaluation

CHANGELOG.md

@@ -779,21 +779,18 @@ Co-authored-by: sachin.sharma <sachin.sharma@juspay.in>
 ### 🎯 Major Feature: Dynamic Model Configuration System
 
 - **⚡ Revolutionary Model Management**: Introduced dynamic model configuration system replacing static enums
-
   - **Self-Updating Models**: New models automatically available without code updates
   - **Cost Optimization**: Automatic selection of cheapest models for tasks
   - **Smart Resolution**: Fuzzy matching, aliases, and capability-based search
   - **Multi-Source Loading**: Configuration from API → GitHub → local with fallback
 
 - **💰 Cost Intelligence**: Built-in cost optimization and model selection algorithms
-
   - **Current Leader**: Gemini 2.0 Flash at $0.000075/1K input tokens
   - **Capability Mapping**: Find models by features (functionCalling, vision, code-execution)
   - **Real-Time Pricing**: Always current model costs and performance data
   - **Budget Controls**: Maximum price filtering and cost-aware selection
 
 - **🔧 Production-Ready Infrastructure**: Complete system with validation and monitoring
-
   - **Model Configuration Server**: REST API with search capabilities (`scripts/model-server.js`)
   - **Zod Schema Validation**: Type-safe runtime configuration validation
   - **Comprehensive Testing**: Full test suite for all dynamic model functionality
@@ -845,19 +842,16 @@ Co-authored-by: sachin.sharma <sachin.sharma@juspay.in>
 ### Bug Fixes - MCP System Restoration
 
 - **🔧 Fixed Built-in Tool Loading**: Resolved critical circular dependency issues preventing default tools from loading
-
   - **Root Cause**: Circular dependency between `config.ts` and `unified-registry.ts` preventing proper initialization
   - **Solution**: Implemented dynamic imports and restructured initialization chain
   - **Result**: Built-in tools restored from 0 → 3 tools (100% recovery rate)
 
 - **⏰ Fixed Time Tool Functionality**: Time tool now properly available and returns accurate real-time data
-
   - Fixed tool registration and execution pathway
   - Proper timezone handling and formatting
   - Verified accuracy against system time
 
 - **🔍 Enhanced External Tool Discovery**: 58+ external MCP tools now discoverable via comprehensive auto-discovery
-
   - Auto-discovery across VS Code, Claude Desktop, Cursor, Windsurf
   - Proper placeholder system for lazy activation
   - Unified registry integration
@@ -912,7 +906,6 @@ Co-authored-by: sachin.sharma <sachin.sharma@juspay.in>
 ### Features
 
 - **🛠️ Enhanced CLI with Ollama Commands**: New Ollama-specific management commands
-
   - `neurolink ollama list-models` - List installed local models
   - `neurolink ollama pull <model>` - Download models locally
   - `neurolink ollama remove <model>` - Remove installed models
@@ -993,14 +986,12 @@ neurolink generate-text "test" --debug
 ### Patch Changes
 
 - **🔧 Production-Ready CLI Logging System**: Fixed critical logging system for clean production output
-
   - **Issue**: CLI showed excessive debug output during normal operation, breaking demo presentations
   - **Root Cause**: Mixed console.log statements bypassed conditional logger system
   - **Solution**: Systematic replacement of all console.log with logger.debug across codebase
   - **Impact**: **Clean CLI output by default** with conditional debug available via `NEUROLINK_DEBUG=true`
 
 - **🔄 Enhanced Provider Fallback Logic**: Fixed incomplete provider fallback coverage
-
   - **Issue**: Provider fallback only attempted 4 of 6 providers (missing Anthropic & Azure)
   - **Root Cause**: Incomplete provider array in NeuroLink class fallback logic
   - **Solution**: Updated to include all 6 providers: `['openai', 'vertex', 'bedrock', 'anthropic', 'azure', 'google-ai']`
@@ -1044,7 +1035,6 @@ NEUROLINK_DEBUG=true node dist/cli/cli/index.js generate-text "test" --max-token
 ### Patch Changes
 
 - **🔧 Critical CLI Dependency Fix**: Removed peer dependencies to ensure zero-friction CLI usage
-
   - **Issue**: CLI commands failed when provider-specific SDK packages were peer dependencies
   - **Root Cause**: `npx` doesn't install peer dependencies, causing missing module errors
   - **Solution**: Moved ALL AI provider SDKs to regular dependencies

docs/CONTEXT-SUMMARIZATION.md

@@ -61,22 +61,18 @@ const neurolink = new NeuroLink({
 The `conversationMemory` configuration object accepts the following properties related to summarization:
 
 - `enableSummarization: boolean`
-
   - **Description**: Set to `true` to enable the automatic summarization feature. `enabled` must also be `true`.
   - **Default**: `false`
 
 - `summarizationThresholdTurns: number`
-
   - **Description**: The number of turns after which summarization should be triggered.
   - **Default**: `20`
 
 - `summarizationTargetTurns: number`
-
   - **Description**: The number of recent turns to _keep_ when a summary is created. The older turns will be replaced by the summary.
   - **Default**: `10`
 
 - `summarizationModel: string`
-
   - **Description**: The specific AI model to use for the summarization task. It's recommended to use a fast and cost-effective model.
   - **Default**: `"gemini-2.5-flash"`
 

docs/DYNAMIC-MODELS.md

@@ -17,13 +17,11 @@ The dynamic model system enables:
 ### Components
 
 1. **Model Configuration Server** (`scripts/model-server.js`)
-
    - Serves model configurations via REST API
    - Provides search and filtering capabilities
    - Can be hosted anywhere (GitHub, CDN, internal server)
 
 2. **Dynamic Model Provider** (`src/lib/core/dynamicModels.ts`)
-
    - Loads configurations from multiple sources with fallback
    - Caches configurations to reduce network requests
    - Validates configurations using Zod schemas

docs/MCP-CONFIGURATION-LOCATIONS.md

@@ -120,13 +120,11 @@ Most tools follow a similar JSON structure:
 
 1. **Common Pattern**: Almost all tools use JSON files with an `mcpServers` object
 2. **Location Hierarchy**: Tools typically check in this order:
-
    - Project/workspace specific configs
    - User/global configs
    - Default/fallback configs
 
 3. **Platform Differences**:
-
    - macOS: Often uses `~/Library/Application Support/`
    - Linux: Typically uses `~/.config/`
    - Windows: Usually uses `%APPDATA%`

docs/PERFORMANCE-OPTIMIZATION.md

@@ -505,14 +505,12 @@ const dashboard = {
 ### Common Issues
 
 1. **High Latency**
-
    - Check provider response times
    - Verify network connectivity
    - Review request complexity
    - Consider request timeouts
 
 2. **Low Throughput**
-
    - Increase connection pool size
    - Enable parallel processing
    - Optimize request batching

docs/REAL-TIME-SPEECH-AGENTS.md

@@ -335,7 +335,6 @@ The subsystem follows the project’s dotenv loading pattern. No hard dependency
 4. Telemetry (OTEL)
    - Add optional spans and metrics; health endpoints.
 5. NeuroLink Client SDK (WS bridge — new package)
-
    - Build a brand-new client SDK as a separate npm package `@juspay/neurolink-client`.
    - Connects to your server’s WS endpoint; no audio capture/playback included.
    - Responsibilities: send upstream audio frames and control messages to server; receive downstream audio/status/text events from server.

docs/TESTING.md

@@ -244,19 +244,16 @@ node ./dist/cli/index.js generate "Help with task" --context '{"userId":"123","d
 ### Common Issues
 
 1. **Empty Responses from Google AI**
-
    - Check model name in .env file
    - Use `gemini-2.5-pro` instead of deprecated models
    - Verify API key is valid
 
 2. **NaN Token Counts**
-
    - Usually indicates provider API failure
    - Check model configuration and API keys
    - Test with `--debug` flag for detailed logs
 
 3. **Enhancement Data Missing**
-
    - Ensure using `--debug` flag to see enhancement output
    - Verify enhancement flags are correctly specified
    - Check that provider is working (not falling back)

docs/TROUBLESHOOTING.md

@@ -1064,7 +1064,6 @@ curl -I --proxy $HTTPS_PROXY https://api.openai.com
 **Solutions**:
 
 1. **Contact IT team** for allowlist:
-
    - `generativelanguage.googleapis.com` (Google AI)
    - `api.anthropic.com` (Anthropic)
    - `api.openai.com` (OpenAI)

docs/advanced/dynamic-models.md

@@ -17,13 +17,11 @@ The dynamic model system enables:
 ### Components
 
 1. **Model Configuration Server** (`scripts/modelServer.js`)
-
    - Serves model configurations via REST API
    - Provides search and filtering capabilities
    - Can be hosted anywhere (GitHub, CDN, internal server)
 
 2. **Dynamic Model Provider** (`src/lib/core/dynamicModels.ts`)
-
    - Loads configurations from multiple sources with fallback
    - Caches configurations to reduce network requests
    - Validates configurations using Zod schemas

docs/analysis/MASTER_NEUROLINK_COMPLETE_ANALYSIS.md

@@ -232,13 +232,11 @@
 **Priority**: Fix verified gaps and improve documentation accuracy
 
 1. **Implement Missing Context Option**
-
    - [ ] Add `--context` option support (VERIFIED as missing)
    - [ ] Ensure context data flows to analytics system
    - [ ] Update help documentation
 
 2. **Fix Remaining Documentation Mismatches**
-
    - [ ] Verify all remaining options (--system, --max-tokens, --quiet)
    - [ ] Test and fix any other flag mismatches
    - [ ] Update examples to use correct flags
@@ -253,7 +251,6 @@
 **Priority**: Implement most critical missing systems
 
 1. **Essential Commands**
-
    - [ ] Implement basic `models list` command (VERIFIED missing)
    - [ ] Add `discover` command for MCP integration (VERIFIED missing)
    - [ ] Complete essential `config` subcommands (setup, show, set) (VERIFIED missing)
@@ -268,7 +265,6 @@
 **Priority**: Complete missing management systems
 
 1. **Models Management System**
-
    - [ ] Implement complete models command system
    - [ ] Add model server at localhost:3001
    - [ ] Implement cost optimization features

docs/analysis/VERIFICATION_RESULTS.md

@@ -2194,23 +2194,20 @@ Use --help to see available options.
 ### 🚨 **TIER 1 CRITICAL ISSUES** (Break Core Functionality):
 
 1. **🔥 TOKEN COUNTING SYSTEM BROKEN**
-
    - **Impact**: CRITICAL - Core analytics feature non-functional
    - **Scope**: 8 of 9 providers affected (only Mistral works)
    - **Root Cause**: Provider-specific token extraction in analytics.ts lines 92-157
    - **Evidence**: Consistent 0/0 input/output tokens across all providers except Mistral
    - **User Impact**: Analytics data meaningless, cost tracking unreliable
 
 2. **🔥 CONTEXT OPTION COMPLETELY IGNORED**
-
    - **Impact**: CRITICAL - Documented feature completely non-functional
    - **Scope**: All CLI usage scenarios
    - **Root Cause**: Context JSON parsing/integration missing
    - **Evidence**: `--context '{"userId":"123"}'` has zero effect on output
    - **User Impact**: Custom analytics tracking impossible
 
 3. **🔥 TOOL USAGE TRACKING BROKEN**
-
    - **Impact**: CRITICAL - Tool analytics completely missing
    - **Scope**: All tool integrations across all providers
    - **Root Cause**: Tool execution tracking system broken
@@ -2227,13 +2224,11 @@ Use --help to see available options.
 ### 🚨 **TIER 2 HIGH IMPACT ISSUES** (Limit Functionality):
 
 5. **⚠️ MCP COMMAND SYSTEM MISSING**
-
    - **Impact**: HIGH - External MCP server management unavailable
    - **Evidence**: No `mcp` commands found in CLI help
    - **User Impact**: Cannot manage external MCP servers via CLI
 
 6. **⚠️ OLLAMA PROVIDER BROKEN**
-
    - **Impact**: HIGH - Local AI provider completely non-functional
    - **Evidence**: Empty responses despite model loading
    - **User Impact**: Local AI workflows impossible
@@ -2250,25 +2245,21 @@ Use --help to see available options.
 ### 🏆 **TIER 1 EXCELLENT IMPLEMENTATIONS**:
 
 1. **🏆 PROVIDER SYSTEM ARCHITECTURE**
-
    - **Quality**: EXCELLENT - 9 providers implemented with unified interface
    - **Evidence**: Professional BaseProvider pattern, comprehensive status checking
    - **Standout**: Automatic provider selection, error handling, performance metrics
 
 2. **🏆 EVALUATION SYSTEM**
-
    - **Quality**: EXCELLENT - Complete 1-10 scoring system
    - **Evidence**: Relevance, accuracy, completeness, overall scores all working
    - **Standout**: Professional evaluation infrastructure with timing and metadata
 
 3. **🏆 BATCH PROCESSING**
-
    - **Quality**: EXCELLENT - Multi-prompt processing with progress indicators
    - **Evidence**: Clean JSON array output, real-time progress spinners
    - **Standout**: Professional UI with completion messages
 
 4. **🏆 JSON OUTPUT SYSTEM**
-
    - **Quality**: EXCELLENT - Comprehensive structured responses
    - **Evidence**: Consistent format across providers, rich metadata
    - **Standout**: Tool availability listing, usage statistics, timing data
@@ -2281,7 +2272,6 @@ Use --help to see available options.
 ### 🎯 **BREAKTHROUGH DISCOVERIES**:
 
 6. **🎯 MISTRAL TOKEN ACCURACY**
-
    - **Discovery**: ONLY provider with accurate token counting
    - **Evidence**: 22 input + 76 output = 98 total (mathematically correct)
    - **Impact**: Proves token counting infrastructure works, isolates bug
@@ -2298,19 +2288,16 @@ Use --help to see available options.
 ### 🚨 **IMMEDIATE CRITICAL FIXES** (Priority 1):
 
 1. **Fix Token Counting for 8 Providers**
-
    - **Location**: `src/lib/core/analytics.ts` lines 92-157
    - **Action**: Study Mistral's working token extraction, apply to other providers
    - **Impact**: Restore core analytics functionality
 
 2. **Implement Context Option Processing**
-
    - **Location**: CLI option parsing and analytics integration
    - **Action**: Parse `--context` JSON and integrate into analytics system
    - **Impact**: Enable custom analytics tracking
 
 3. **Fix Tool Usage Tracking**
-
    - **Location**: Tool execution and analytics integration
    - **Action**: Ensure tool calls are recorded in `toolsUsed` array
    - **Impact**: Restore tool analytics visibility
@@ -2323,12 +2310,10 @@ Use --help to see available options.
 ### 🔧 **HIGH PRIORITY ENHANCEMENTS** (Priority 2):
 
 5. **Implement MCP CLI Commands**
-
    - **Action**: Add mcp list, install, test, exec, remove commands
    - **Impact**: Enable external MCP server management
 
 6. **Fix Ollama Provider**
-
    - **Action**: Debug empty response issue in Ollama integration
    - **Impact**: Restore local AI functionality
 
@@ -2339,12 +2324,10 @@ Use --help to see available options.
 ### 🎨 **QUALITY IMPROVEMENTS** (Priority 3):
 
 8. **Fix Provider Tool Confusion**
-
    - **Action**: Clarify tool availability messaging for Anthropic/Vertex
    - **Impact**: Accurate tool capability reporting
 
 9. **Enhance Evaluation Reasoning**
-
    - **Action**: Ensure evaluation reasoning field is populated
    - **Impact**: Better evaluation transparency