Skip to content

feat(proxy): comprehensive proxy support for all AI providers - #124

Merged
murdore merged 1 commit into
releasefrom
feat/proxy-support-comprehensive
Aug 25, 2025
Merged

murdore merged 1 commit into
releasefrom
feat/proxy-support-comprehensive

Conversation

@murdore

@murdore murdore commented Aug 25, 2025 •

Copy link
Copy Markdown
Contributor
  • Enhanced Proxy Infrastructure: Comprehensive proxy configuration system with exhaustive environment variable detection and logging
  • Universal Provider Integration: All AI providers (Vertex, Bedrock, OpenAI-Compatible, LiteLLM) now support proxy connections via createProxyFetch()
  • Robust Authentication: Improved Google Cloud authentication with dual methods (service account files + individual env vars) and proper file existence validation
  • Enhanced Health Monitoring: Provider health checks now include proxy-aware connectivity testing and comprehensive validation
  • ESM Compatibility: Resolved CommonJS/ESM compatibility issues for Node.js module imports
  • Extensive Logging: Detailed proxy configuration and request tracking for debugging corporate firewall environments

Pull Request

Description

Type of Change

  • [.] 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🧹 Code refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test coverage improvement
  • 🔧 Build/CI configuration change

Related Issues

  • Fixes #
  • Related to #

Changes Made

AI Provider Impact

  • OpenAI
  • Anthropic
  • Google AI/Vertex
  • AWS Bedrock
  • Azure OpenAI
  • Hugging Face
  • Ollama
  • Mistral
  • [.] All providers
  • No provider-specific changes

Component Impact

  • [.] CLI
  • [.] SDK
  • [.] MCP Integration
  • Streaming
  • Tool Calling
  • Configuration
  • Documentation
  • Tests

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • E2E tests added/updated
  • Manual testing performed
  • All existing tests pass

Test Environment

  • OS:
  • Node.js version:
  • Package manager:

Performance Impact

  • No performance impact
  • Performance improvement
  • Minor performance impact (acceptable)
  • Significant performance impact (needs discussion)

Breaking Changes

Screenshots/Demo

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

Summary by CodeRabbit

  • New Features

    • In-memory MCP server management APIs in NeuroLink.
    • Google Vertex now supports Anthropic (Claude) models and provides cache controls.
    • Background provider health checks and a detailed Vertex readiness checker.
  • Improvements

    • Proxy-based networking for OpenAI-compatible, LiteLLM, and Bedrock providers.
    • More robust Google Vertex authentication/configuration and model routing.
    • Expanded Vertex model availability reporting.
  • Refactor

    • Extensive diagnostics and telemetry across NeuroLink, Vertex, streaming, and proxy fetch for easier troubleshooting.
  • Style

    • Minor formatting updates with no behavioral changes.

@murdore
murdore requested a review from Copilot August 25, 2025 10:04
@coderabbitai

coderabbitai Bot commented Aug 25, 2025 •

Copy link
Copy Markdown

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Adds extensive runtime instrumentation across NeuroLink, Google Vertex, proxy fetch, and provider layers; introduces proxy-based HTTP routing; adds Anthropic-on-Vertex routing and validation; updates provider health checks with Vertex/Anthropic support; adds new NeuroLink in-memory MCP server APIs; adjusts several method signatures (mostly parameter naming/async) and removes redundant generation methods.

Changes

Cohort / File(s) Summary
Tool discovery logging format
src/lib/mcp/toolDiscoveryService.ts
Reformats debug logging in validateToolOutput; no behavior changes.
NeuroLink instrumentation & MCP APIs
src/lib/neurolink.ts
Adds granular telemetry (C/M/G/A log points), staged initialization, external MCP bootstrap, memory instrumentation, and new public APIs: addInMemoryMCPServer, getInMemoryServers, getInMemoryServerInfos, getAutoDiscoveredServerInfos. No signature changes to existing public methods.
Proxy fetch integration
src/lib/proxy/proxyFetch.ts
Adds exhaustive logging around proxy selection, undici import, URL analysis, dispatcher creation, fetch result, and error fallback; no API changes.
Amazon Bedrock provider refactor
src/lib/providers/amazonBedrock.ts
Introduces proxy fetch into AWS client config; narrows executeStream param type (unused); removes executeGenerate (handled by BaseProvider); trims imports/types.
OpenAI-compatible provider proxying
src/lib/providers/openaiCompatible.ts
Routes client and model listing via createProxyFetch; renames unused executeStream param; cleans up imports.
LiteLLM provider proxying
src/lib/providers/litellm.ts
Injects createProxyFetch into client and model listing; removes direct OpenAI SDK usage; renames unused executeStream param; cleans imports.
Google Vertex dual-path (Vertex + Anthropic)
src/lib/providers/googleVertex.ts
Adds Anthropic-on-Vertex path with async settings, runtime credentials generation, validation, and detailed logging; new static cache utils; updated constructor param name; async model creation helpers.
Provider health checks (Vertex/Anthropic)
src/lib/utils/providerHealth.ts
Adds background health checks; Vertex dual-auth validation; Anthropic-on-Vertex support probe (public static), connectivity/regional checks, expanded model lists; note: duplicate insertion of checkVertexAnthropicSupport in diff.
Build system formatting
tools/automation/buildSystem.js
Formatting-only edits; no functional changes.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant NL as NeuroLink
  participant PR as ProviderRegistry
  participant MCP as External MCP Server
  participant MM as MemoryManager (dyn import)

  NL->>NL: Constructor start (C001)
  NL->>PR: Configure providers (C002)
  PR-->>NL: Providers configured (C004)
  NL->>MCP: Init external server (C009)
  MCP-->>NL: Setup complete (C010–C012)
  NL->>MM: Dynamic import (M004)
  MM-->>NL: Import result (M005)
  NL-->>NL: Telemetry checkpoints (M007–M015)
Loading
sequenceDiagram
  autonumber
  participant GV as GoogleVertexProvider
  participant CFG as createVertexSettings (async)
  participant ANT as createVertexAnthropic
  participant GML as Google Vertex Models
  participant AML as Vertex Anthropic Models

  GV->>CFG: Build settings (auth/env) [V003–V007]
  CFG-->>GV: Settings (project, region, creds)
  GV->>GV: isAnthropicModel?
  alt Anthropic
    GV->>ANT: Validate & create Anthropic model [V008–V012]
    ANT-->>GV: LanguageModelV1 or null
    GV->>AML: Stream/generate if model created
  else Google
    GV->>GML: Create Google Vertex model [V013–V014]
  end
Loading
sequenceDiagram
  autonumber
  participant CL as Client
  participant PF as createProxyFetch
  participant UD as undici.fetch
  participant OR as Origin

  CL->>PF: fetch(input, init)
  PF->>PF: Analyze env/proxy/URL
  alt Proxy available
    PF->>UD: fetch with ProxyAgent dispatcher
  else No proxy
    PF->>UD: fetch without dispatcher
  end
  UD->>OR: HTTP request
  OR-->>UD: Response
  UD-->>PF: Response
  PF-->>CL: Response (or fallback to global fetch on error)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

released

Poem

I tap my paws on proxy trails, hop-hop through logs so bright—
Claude meets Vertex, stars align, we route by moonlit night.
Neurolink hums with metrics fine, MCPs in tidy rows,
Health checks nibble edge to edge, where gentle carrot grows.
Deploy, reflect—then thump with joy: the streams, they smoothly flow. 🥕✨

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/proxy-support-comprehensive

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions

github-actions Bot commented Aug 25, 2025 •

Copy link
Copy Markdown
Contributor

✅ Single Commit Policy - COMPLIANT

Status: Policy requirements met • 1 commit • Valid format • Ready for merge

📊 View validation details

📝 Commit Details

  • Hash: 9a4e8e100b678f04fe3c0b7fdf28dbfabb160cb7
  • Message: feat(proxy): comprehensive proxy support for all AI providers
  • Author: Sachin Sharma

✅ Validation Results

  • Single commit requirement met
  • No merge commits in branch
  • Semantic commit message format verified
  • Ready for squash merge to release branch

🤖 Automated validation by NeuroLink Single Commit Enforcement

@github-actions

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

Comment thread src/lib/providers/googleVertex.ts Dismissed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces comprehensive proxy support across all AI providers, enhanced authentication mechanisms for Google Cloud services, and extensive logging infrastructure for debugging corporate firewall environments.

Key Changes:

  • Universal Proxy Integration: All AI providers now support proxy connections via createProxyFetch() function
  • Enhanced Google Vertex Authentication: Dual authentication methods with file existence validation and runtime credentials file creation
  • Comprehensive Logging: Exhaustive debug logging for proxy configurations, network requests, and authentication flows

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
tools/automation/buildSystem.js Code formatting improvements with multi-line function calls
src/lib/utils/providerHealth.ts Enhanced Vertex AI authentication validation and comprehensive Anthropic support checking
src/lib/proxy/proxyFetch.ts Detailed proxy request logging with comprehensive error analysis
src/lib/providers/openaiCompatible.ts Proxy support integration and code cleanup
src/lib/providers/litellm.ts Proxy support integration and unused import removal
src/lib/providers/googleVertex.ts Major authentication overhaul with dual provider architecture and extensive logging
src/lib/providers/amazonBedrock.ts Proxy support integration
src/lib/neurolink.ts Comprehensive initialization and stream processing logging
src/lib/mcp/toolDiscoveryService.ts Minor formatting improvements

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread src/lib/providers/googleVertex.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (5)
src/lib/mcp/toolDiscoveryService.ts (1)

751-757: Avoid logging entire tool outputs; redact and truncate to prevent sensitive data leakage and log bloat

fullResponse: result can dump PII, API tokens, or large payloads to logs. Redact common secret-looking keys and cap size.

-    mcpLogger.debug("[ToolDiscoveryService] Tool response received", {
-      type: typeof result,
-      isArray: Array.isArray(result),
-      isObject: isObject(result),
-      hasKeys: isObject(result) ? Object.keys(result as object).length : 0,
-      fullResponse: result, // Log the complete response, not a truncated sample
-    });
+    // Redact and limit payload to avoid leaking sensitive data or flooding logs
+    const safePreview = (() => {
+      try {
+        const MAX = 2048;
+        const json =
+          typeof result === "string"
+            ? result
+            : JSON.stringify(result, (_k, v) => {
+                // redact common secret-like fields
+                if (
+                  typeof _k === "string" &&
+                  /token|secret|authorization|api[_-]?key|password/i.test(_k)
+                ) {
+                  return "[REDACTED]";
+                }
+                return v;
+              });
+        return json.length > MAX ? json.slice(0, MAX) + "…[truncated]" : json;
+      } catch {
+        return String(result).slice(0, 512);
+      }
+    })();
+    mcpLogger.debug("[ToolDiscoveryService] Tool response received", {
+      type: typeof result,
+      isArray: Array.isArray(result),
+      isObject: isObject(result),
+      hasKeys: isObject(result) ? Object.keys(result as object).length : 0,
+      preview: safePreview,
+    });

Optionally, extract the redaction logic into a shared utility if used elsewhere.

src/lib/neurolink.ts (1)

1391-1527: MCP retry loop double-increments attempts; reduces total retries and misreports attempt numbers

mcpAttempts is incremented in the catch and again after the try/catch, causing off-by-one behavior and fewer actual attempts than configured. It also logs incorrect attempt indices. Refactor to a for-loop or ensure a single increment per iteration.

-        while (mcpAttempts <= maxMcpRetries) {
-          try {
+        const maxAttempts = maxMcpRetries + 1;
+        for (let attempt = 1; attempt <= maxAttempts; attempt++) {
+          try {
             // 🚀 EXHAUSTIVE LOGGING POINT G007: MCP GENERATION ATTEMPT
             const mcpAttemptStartTime = process.hrtime.bigint();
             logger.debug(`[NeuroLink] 🎯 LOG_POINT_G007_MCP_ATTEMPT_START`, {
               logPoint: "G007_MCP_ATTEMPT_START",
               generateInternalId,
               timestamp: new Date().toISOString(),
               elapsedMs: Date.now() - generateInternalStartTime,
               elapsedNs: (
                 process.hrtime.bigint() - generateInternalHrTimeStart
               ).toString(),
               mcpAttemptStartTimeNs: mcpAttemptStartTime.toString(),
-              currentAttempt: mcpAttempts + 1,
-              maxAttempts: maxMcpRetries + 1,
-              isFirstAttempt: mcpAttempts === 0,
-              isLastAttempt: mcpAttempts === maxMcpRetries,
-              attemptType: mcpAttempts === 0 ? "INITIAL" : "RETRY",
-              message: `Attempting MCP generation (attempt ${mcpAttempts + 1}/${maxMcpRetries + 1})`,
+              currentAttempt: attempt,
+              maxAttempts,
+              isFirstAttempt: attempt === 1,
+              isLastAttempt: attempt === maxAttempts,
+              attemptType: attempt === 1 ? "INITIAL" : "RETRY",
+              message: `Attempting MCP generation (attempt ${attempt}/${maxAttempts})`,
             });
             logger.debug(
-              `[${functionTag}] Attempting MCP generation (attempt ${mcpAttempts + 1}/${maxMcpRetries + 1})...`,
+              `[${functionTag}] Attempting MCP generation (attempt ${attempt}/${maxAttempts})...`,
             );
             const mcpResult = await this.tryMCPGeneration(options);
             // … unchanged …
             if (mcpResult && mcpResult.content) {
               // success path …
               return mcpResult;
             } else {
               // analyze empty content path …
               if (
                 mcpResult &&
                 mcpResult.toolExecutions &&
                 mcpResult.toolExecutions.length > 0
               ) {
                 // success-with-tools path …
                 return mcpResult;
               }
             }
-          } catch (error) {
-            mcpAttempts++;
+          } catch (error) {
             logger.debug(
-              `[${functionTag}] MCP generation failed on attempt ${mcpAttempts}/${maxMcpRetries + 1}`,
+              `[${functionTag}] MCP generation failed on attempt ${attempt}/${maxAttempts}`,
               {
                 error: error instanceof Error ? error.message : String(error),
-                willRetry: mcpAttempts <= maxMcpRetries,
+                willRetry: attempt < maxAttempts,
               },
             );
-            // If this was the last attempt, break and fall back
-            if (mcpAttempts > maxMcpRetries) {
+            // If this was the last attempt, break and fall back
+            if (attempt >= maxAttempts) {
               logger.debug(
                 `[${functionTag}] All MCP attempts exhausted, falling back to direct generation`,
               );
               break;
             }
             // Small delay before retry to allow transient issues to resolve
             await new Promise((resolve) => setTimeout(resolve, 500));
-          }
-
-          mcpAttempts++;
-        }
+          }
+        }
src/lib/utils/providerHealth.ts (1)

1398-1412: Include Azure in health checks

The default preferredProviders in getBestHealthyProvider includes "azure", but checkAllProvidersHealth only iterates over:

  • AIProviderName.ANTHROPIC
  • AIProviderName.OPENAI
  • AIProviderName.VERTEX
  • AIProviderName.GOOGLE_AI
  • AIProviderName.BEDROCK
  • AIProviderName.OLLAMA

As a result, Azure will never be health-checked and may be returned without verification. Since Azure is a first-class, registered provider (see providerRegistry and AzureOpenAIProvider implementation), we should add it to the health-check list.

• File: src/lib/utils/providerHealth.ts (around line 1451)
• Modify the providers array in checkAllProvidersHealth:

 const providers: AIProviderName[] = [
   AIProviderName.ANTHROPIC,
   AIProviderName.OPENAI,
   AIProviderName.VERTEX,
   AIProviderName.GOOGLE_AI,
   AIProviderName.BEDROCK,
+  AIProviderName.AZURE,
   AIProviderName.OLLAMA,
 ];

If Azure is intentionally excluded from quick health checks, instead remove "azure" from the default list in getBestHealthyProvider. But given Azure’s full implementation elsewhere, adding it here ensures consistency.

src/lib/proxy/proxyFetch.ts (1)

149-155: Preserve Request semantics; don’t coerce input to string

Coercing input to string breaks Request objects (method/body/headers). Pass input through unchanged.

Apply this diff:

-        const response = await undici.fetch(
-          input as string,
-          {
+        const response = await undici.fetch(
+          input as any,
+          {
             ...init,
             dispatcher: dispatcher,
           } as unknown as import("undici").RequestInit,
         );
src/lib/providers/amazonBedrock.ts (1)

24-27: Remove account-specific ARN default; it will break for all other accounts

The fallback to arn:aws:bedrock:...:225681119357:... hard-codes a specific account’s inference profile and will fail universally.

Use an env-only configuration or a broadly available model identifier. Example fix:

@@
-  return (
-    process.env.BEDROCK_MODEL ||
-    process.env.BEDROCK_MODEL_ID ||
-    "arn:aws:bedrock:us-east-2:225681119357:inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0"
-  );
+  const model =
+    process.env.BEDROCK_MODEL || process.env.BEDROCK_MODEL_ID;
+  if (!model) {
+    throw new Error(
+      "BEDROCK_MODEL (or BEDROCK_MODEL_ID) is required. Example: 'anthropic.claude-3-haiku-20240307-v1:0' or a valid inference profile ARN.",
+    );
+  }
+  return model;
🧹 Nitpick comments (14)
src/lib/mcp/toolDiscoveryService.ts (1)

741-748: Comment and behavior mismatch: we never throw on null/undefined results

The comment says "Only throw for truly invalid responses (null/undefined)" but the code logs and returns without throwing. Either fix the comment or introduce a throw. Given the intended permissive behavior, I'd update the comment to avoid confusion.

-    // Only throw for truly invalid responses (null/undefined)
+    // For null/undefined responses, do not throw; treat as empty and let the AI handle it
     if (isNullish(result)) {
       mcpLogger.debug(
         "[ToolDiscoveryService] Tool returned null/undefined, treating as empty response",
       );
       // Even null responses can be valid for some tools - don't throw
       return;
     }
tools/automation/buildSystem.js (1)

380-382: Nit: prefer a function to echo the skip-build message for consistency/testing

Current constant is fine; if you later add color or localization, a helper would centralize it. No change required now.

src/lib/neurolink.ts (1)

631-649: Optional: make initializeMCP concurrency-safe

If generate() and stream() call initializeMCP concurrently, both paths may run heavy initialization. Guard with a shared in-flight promise.

Sketch:

// class field
private mcpInitInFlight?: Promise<void>;

// in initializeMCP() entry
if (this.mcpInitialized) return;
if (this.mcpInitInFlight) { await this.mcpInitInFlight; return; }
this.mcpInitInFlight = (async () => {
  try {
    // existing initialization …
    this.mcpInitialized = true;
  } finally {
    this.mcpInitInFlight = undefined;
  }
})();
await this.mcpInitInFlight;
src/lib/providers/googleVertex.ts (3)

156-175: Auth fallback flow is solid, but consider early-returning after principal-file success and reducing duplicate env parsing

You already return early if the ADC file exists. For readability and to avoid parsing the env twice, consider structuring as: (1) ADC file exists → return; (2) individual env creds → return settings with googleAuthOptions; (3) final warn.

Also applies to: 176-237


585-656: Create Vertex instance/model with proxy-aware fetch is correct; add small hardening for network diagnostics

Current diagnostics include DNS servers and interfaces; that’s very verbose. Keep at debug level (as you do) and consider gating behind an env flag to reduce noise in prod. No code change required now.

Also applies to: 657-788


59-64: Model default string may drift; consider centralizing latest default in config

Hardcoding "gemini-2.5-flash" is fine today. If you already have a provider config manager (you do), consider pulling the default from there to avoid future churn.

src/lib/utils/providerHealth.ts (2)

486-496: Clarify Vertex API key env var lookup to avoid misleading consumers

getApiKeyEnvironmentVariable returns "GOOGLE_APPLICATION_CREDENTIALS" for Vertex, but Vertex auth supports both a credentials file and GOOGLE_AUTH_CLIENT_EMAIL/GOOGLE_AUTH_PRIVATE_KEY. Returning a single var can mislead downstream code/tools.

Two options:

  • Return an empty string for Vertex and rely on the Vertex-specific branch in checkApiKeyValidity.
  • Or return a descriptive pseudo-key like "VERTEX_AUTH" and update any messaging to explain both auth paths.

1238-1248: Hardcoded regional support list will drift; consider single source of truth

The supportedRegions array will stale over time. Pulling this list from a config module or an env override avoids frequent code changes and mismatches.

Expose supported regions in a constants file or accept a comma-separated env var (VERTEX_ANTHROPIC_SUPPORTED_REGIONS) and default to the current list.

src/lib/providers/openaiCompatible.ts (2)

90-91: Avoid re-instantiating ProxyAgent per call; reuse a single proxy-aware fetch

createProxyFetch() builds a ProxyAgent; creating a new one per request adds overhead. Reuse a single instance across the provider.

Apply this diff:

@@
-    this.customOpenAI = createOpenAI({
+    this.customOpenAI = createOpenAI({
       baseURL: this.config.baseURL,
       apiKey: this.config.apiKey,
-      fetch: createProxyFetch(),
+      fetch: this.proxyFetch,
     });
@@
-      const proxyFetch = createProxyFetch();
-      const response = await proxyFetch(modelsUrl, {
+      const response = await this.proxyFetch(modelsUrl, {
         headers: {
           Authorization: `Bearer ${this.config.apiKey}`,
           "Content-Type": "application/json",
         },
       });

And add this class field near other fields:

private readonly proxyFetch: typeof fetch = createProxyFetch();

Also applies to: 300-302


141-145: Auto-discovery picks the first model; consider preferring stable/default models

Selecting the first returned model can be non-deterministic across providers. If the list includes embeddings or legacy models first, users get surprising results.

Prefer a small allowlist (e.g., gpt-4o, gpt-4o-mini, gpt-3.5-turbo) if present; else fall back to current behavior.

src/lib/providers/litellm.ts (2)

67-68: Reuse a single proxy-aware fetch instead of creating one per request

Same rationale as openaiCompatible: reduce overhead by reusing one ProxyAgent.

Apply this diff:

@@
-    const customOpenAI = createOpenAI({
+    const customOpenAI = createOpenAI({
       baseURL: config.baseURL,
       apiKey: config.apiKey,
-      fetch: createProxyFetch(),
+      fetch: this.proxyFetch,
     });
@@
-      const proxyFetch = createProxyFetch();
-      const response = await proxyFetch(modelsUrl, {
+      const response = await this.proxyFetch(modelsUrl, {
         method: "GET",
         headers: {
           Authorization: `Bearer ${config.apiKey}`,
           "Content-Type": "application/json",
         },
         signal: controller.signal,
       });

Add this class field alongside private model: LanguageModelV1;:

private readonly proxyFetch: typeof fetch = createProxyFetch();

Also applies to: 303-305


271-280: Make fallback models configurable for easier ops overrides

You already read LITELLM_FALLBACK_MODELS; consider logging the env source and trimming/validating entries to reduce surprise at runtime.

No code change required; optional: log which source (env vs default) is used and ignore empty strings after split/trim.

src/lib/proxy/proxyFetch.ts (2)

54-57: Honor NO_PROXY and ALL_PROXY for correctness in mixed-network environments

Current selection ignores NO_PROXY and ALL_PROXY. Respecting them avoids proxying localhost/cluster endpoints and supports a single ALL_PROXY setting.

Sketch:

const allProxy = process.env.ALL_PROXY || process.env.all_proxy;
const noProxy = (process.env.NO_PROXY || process.env.no_proxy || "")
  .split(",")
  .map((s) => s.trim())
  .filter(Boolean);

const shouldBypass = noProxy.some((pattern) => url.hostname.endsWith(pattern));
const selected = shouldBypass
  ? undefined
  : (url.protocol === "https:" ? httpsProxy : httpProxy) || allProxy;

Then use selected for ProxyAgent choice.


114-175: Cache ProxyAgent per proxy URL to reduce overhead

Creating a ProxyAgent for every request is unnecessary. Memoize per proxyUrl.

Minimal change:

const agentCache = new Map<string, import("undici").ProxyAgent>();
// ...
let dispatcher = agentCache.get(proxyUrl);
if (!dispatcher) {
  dispatcher = new ProxyAgent(proxyUrl);
  agentCache.set(proxyUrl, dispatcher);
}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 4a60221 and c745e7d.

📒 Files selected for processing (9)
  • src/lib/mcp/toolDiscoveryService.ts (2 hunks)
  • src/lib/neurolink.ts (9 hunks)
  • src/lib/providers/amazonBedrock.ts (4 hunks)
  • src/lib/providers/googleVertex.ts (8 hunks)
  • src/lib/providers/litellm.ts (5 hunks)
  • src/lib/providers/openaiCompatible.ts (5 hunks)
  • src/lib/proxy/proxyFetch.ts (4 hunks)
  • src/lib/utils/providerHealth.ts (9 hunks)
  • tools/automation/buildSystem.js (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (8)
src/lib/mcp/toolDiscoveryService.ts (1)
src/lib/utils/typeUtils.ts (1)
  • isNullish (97-99)
src/lib/proxy/proxyFetch.ts (2)
src/lib/utils/logger.ts (2)
  • logger (198-237)
  • error (134-136)
tools/testing/proxy_server.js (1)
  • URL (29-29)
src/lib/providers/litellm.ts (1)
src/lib/proxy/proxyFetch.ts (1)
  • createProxyFetch (12-209)
src/lib/providers/openaiCompatible.ts (2)
src/lib/proxy/proxyFetch.ts (1)
  • createProxyFetch (12-209)
src/lib/types/typeAliases.ts (1)
  • ZodUnknownSchema (18-18)
src/lib/providers/amazonBedrock.ts (4)
src/lib/proxy/proxyFetch.ts (1)
  • createProxyFetch (12-209)
src/lib/types/typeAliases.ts (1)
  • ZodUnknownSchema (18-18)
src/lib/types/streamTypes.ts (1)
  • StreamResult (144-178)
src/lib/utils/messageBuilder.ts (1)
  • buildMessagesArray (16-64)
src/lib/neurolink.ts (9)
src/lib/utils/logger.ts (3)
  • logger (198-237)
  • error (134-136)
  • mcpLogger (240-240)
src/lib/factories/providerRegistry.ts (1)
  • ProviderRegistry (26-276)
src/lib/utils/conversationMemoryUtils.ts (1)
  • applyConversationMemoryDefaults (22-33)
src/lib/core/conversationMemoryManager.ts (1)
  • ConversationMemoryManager (20-214)
src/lib/mcp/externalServerManager.ts (1)
  • ExternalServerManager (155-1468)
src/lib/utils/performance.ts (1)
  • MemoryManager (109-185)
src/lib/mcp/toolRegistry.ts (2)
  • toolRegistry (827-827)
  • ToolInfo (831-831)
src/lib/mcp/servers/agent/directToolsServer.ts (1)
  • directToolsServer (17-23)
src/lib/mcp/contracts/mcpContract.ts (1)
  • ToolInfo (50-58)
src/lib/utils/providerHealth.ts (2)
src/lib/index.ts (1)
  • AIProviderName (15-15)
src/lib/utils/logger.ts (2)
  • logger (198-237)
  • error (134-136)
src/lib/providers/googleVertex.ts (4)
src/lib/proxy/proxyFetch.ts (1)
  • createProxyFetch (12-209)
src/lib/utils/logger.ts (2)
  • logger (198-237)
  • error (134-136)
src/lib/utils/timeout.ts (1)
  • createTimeoutController (436-469)
src/lib/utils/messageBuilder.ts (1)
  • buildMessagesArray (16-64)
🔇 Additional comments (5)
tools/automation/buildSystem.js (2)

350-353: LGTM: clearer deploy command construction with explicit default for forceRebuild

Moving the boolean default and formatting the call improves readability without changing behavior.


468-470: LGTM: CLI usage text wrapping improves readability

User help is clearer; no behavior changes.

src/lib/providers/googleVertex.ts (2)

365-410: Dual-path model routing and instrumentation look good

Anthropic-on-Vertex path selection and detailed logging are helpful for ops. The fallback to Google model when Anthropic is unavailable is sensible.

Also applies to: 432-554


869-982: Stream timeout handling is consistent; fix missing AbortSignal in AmazonBedrockProvider

The other providers correctly pass abortSignal (via timeoutController.controller.signal) down into streamText or HTTP fetch calls, but in AmazonBedrockProvider.executeStream we neither set up a timeout controller nor pass an abortSignal to streamText. As a result, Bedrock streams can’t be aborted by the user or timeout.

Locations to update:

  • src/lib/providers/amazonBedrock.ts

Suggested diff:

--- a/src/lib/providers/amazonBedrock.ts
+++ b/src/lib/providers/amazonBedrock.ts
@@ protected async executeStream(
-      const result = streamText({
-        model: this.model,
-        messages: messages,
-        maxTokens: options.maxTokens || DEFAULT_MAX_TOKENS,
-        temperature: options.temperature,
-      });
+      // 🚀 Add timeout controller & propagate abortSignal
+      const timeout = this.getTimeout(options);
+      const timeoutController = createTimeoutController(
+        timeout,
+        this.providerName,
+        "stream",
+      );
+      const result = streamText({
+        model: this.model,
+        messages: messages,
+        maxTokens: options.maxTokens || DEFAULT_MAX_TOKENS,
+        temperature: options.temperature,
+        abortSignal: timeoutController.controller.signal,
+      });
+      timeoutController.cleanup();

Please apply this change to ensure the Bedrock stream can be properly aborted.

Likely an incorrect or invalid review comment.

src/lib/providers/amazonBedrock.ts (1)

71-76: Good: Bedrock client wired to proxy-aware fetch

Wiring fetch: createProxyFetch() into the AWS client aligns with the PR objective and will help customers behind corporate proxies.

Comment thread src/lib/neurolink.ts
Comment thread src/lib/providers/amazonBedrock.ts Outdated
Comment thread src/lib/providers/googleVertex.ts
Comment thread src/lib/providers/googleVertex.ts
Comment thread src/lib/proxy/proxyFetch.ts
Comment thread src/lib/proxy/proxyFetch.ts
Comment thread src/lib/proxy/proxyFetch.ts
Comment thread src/lib/proxy/proxyFetch.ts
Comment thread src/lib/utils/providerHealth.ts Outdated
Comment thread src/lib/utils/providerHealth.ts Outdated
@murdore
murdore force-pushed the feat/proxy-support-comprehensive branch from c745e7d to 74326a1 Compare August 25, 2025 11:28
@github-actions

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

@murdore
murdore force-pushed the feat/proxy-support-comprehensive branch from 74326a1 to 49c5ed7 Compare August 25, 2025 11:31
@github-actions

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

- **Enhanced Proxy Infrastructure**: Comprehensive proxy configuration system with exhaustive environment variable detection and logging
- **Universal Provider Integration**: All AI providers (Vertex, Bedrock, OpenAI-Compatible, LiteLLM) now support proxy connections via createProxyFetch()
- **Robust Authentication**: Improved Google Cloud authentication with dual methods (service account files + individual env vars) and proper file existence validation
- **Enhanced Health Monitoring**: Provider health checks now include proxy-aware connectivity testing and comprehensive validation
- **ESM Compatibility**: Resolved CommonJS/ESM compatibility issues for Node.js module imports
- **Extensive Logging**: Detailed proxy configuration and request tracking for debugging corporate firewall environments
@murdore
murdore force-pushed the feat/proxy-support-comprehensive branch from 49c5ed7 to 9a4e8e1 Compare August 25, 2025 11:46
@github-actions

Copy link
Copy Markdown
Contributor

🤖 AI Review & Build Compliance ✅

Status: AI analysis complete • Build rules validated • Ready for review

📊 View detailed analysis results

🛡️ Analysis Complete

  • ✅ Security scan (vulnerabilities, API keys)
  • ✅ TypeScript safety & code quality
  • ✅ Error handling & best practices
  • ✅ Build rule enforcement validated
  • ✅ Commit format & compliance checks

📋 Ready for Merge When

  • All CI checks passing
  • Manual review approved
  • Any AI-flagged issues resolved

🤖 AI analysis complete - check individual code comments for specific feedback

@murdore
murdore merged commit 332974a into release Aug 25, 2025
11 checks passed
@murdore
murdore deleted the feat/proxy-support-comprehensive branch August 25, 2025 12:03
@murdore
murdore removed the request for review from Copilot March 25, 2026 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants