doc(sdk): Add documentation for model orchestration

[sahiltyagiii]

Pull Request

Description

Created documentation markdown file for advanced model and provider orchestration.

Type of Change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• ✨ New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📚 Documentation update
• 🧹 Code refactoring (no functional changes)
• ⚡ Performance improvement
• 🧪 Test coverage improvement
• 🔧 Build/CI configuration change

Related Issues

None

• Fixes #
• Related to #

AI Provider Impact

None, only documentation update.

• OpenAI
• Anthropic
• Google AI/Vertex
• AWS Bedrock
• Azure OpenAI
• Hugging Face
• Ollama
• Mistral
• All providers
• No provider-specific changes

Component Impact

• CLI
• SDK
• MCP Integration
• Streaming
• Tool Calling
• Configuration
• Documentation
• Tests

Testing

• Unit tests added/updated
• Integration tests added/updated
• E2E tests added/updated
• Manual testing performed
• All existing tests pass

Test Environment

• OS:
• Node.js version:
• Package manager:

Performance Impact

• No performance impact
• Performance improvement
• Minor performance impact (acceptable)
• Significant performance impact (needs discussion)

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published

Summary by CodeRabbit

• New Features

  • Introduces a Smart Orchestration layer that intelligently routes requests to the best provider-model based on task, cost, performance, and health.
  • Adds real-time health monitoring and health-based routing with graceful fallback.
  • Provides new configuration controls and runtime toggles to enable/disable orchestration and tune constraint weights.
  • Exposes orchestration statistics and provider health summaries for visibility.

• Documentation

  • Adds an Advanced Orchestration Implementation Plan with architecture, decision flow, configuration presets, and error-handling guidance.

• Refactor

  • Integrates orchestration into existing generation and streaming workflows to improve reliability and performance.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Introduces a Smart Orchestration layer with new modules, types, config, and data to enable provider-model routing based on task analysis, constraints, and health. Updates NeuroLink and ProviderFactory public APIs to integrate orchestration, health monitoring, and metrics. Adds an advanced planning document detailing architecture, flow, and configuration.

Changes

Cohort / File(s)	Summary of Changes
Documentation docs/ADVANCED-ORCHESTRATION-PLAN.md	Adds a detailed plan outlining architecture, flow, components, configuration, API impacts, and error handling for advanced orchestration.
Orchestration Core src/lib/orchestration/SmartOrchestrator.ts, src/lib/orchestration/TaskClassifier.ts, src/lib/orchestration/LightweightClassifier.ts, src/lib/orchestration/ResearchDataManager.ts, src/lib/orchestration/ConstraintOptimizer.ts, src/lib/orchestration/HealthMonitor.ts, src/lib/orchestration/ConfigurationManager.ts, src/lib/orchestration/index.ts	Introduces orchestration components (routing, classification, research data access, constraint optimization, health monitoring, configuration) and public index exports.
Types & Config src/lib/types/orchestration.ts, src/lib/config/orchestrationConfig.ts	Adds orchestration-related types (configs, stats, health, weights) and configuration surface.
Research Data src/lib/data/modelResearch.ts	Adds pre-researched model performance dataset for routing.
Core API Integration .../NeuroLink.ts	Updates constructor, integrates orchestrator into generate()/stream(), and adds orchestration and health public methods (config, stats, controls).
Provider Factory Integration src/lib/factories/providerFactory.ts	Modifies factory to support health tracking, metrics collection, and integration with health monitoring.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    actor Client
    participant NL as NeuroLink (Core)
    participant SO as SmartOrchestrator
    participant LC as LightweightClassifier
    participant TC as TaskClassifier
    participant CO as ConstraintOptimizer
    participant PF as ProviderFactory
    participant AP as AI Provider
    participant HM as HealthMonitor

    Client->>NL: generate()/stream()(request, config)
    NL->>SO: route(request, smartOrchestrationConfig)
    SO->>LC: quick classify(request)
    alt needs deep classification
        SO->>TC: analyze task
        TC-->>SO: task profile
    else
        LC-->>SO: task profile (light)
    end
    SO->>CO: apply constraints (cost, latency, policy)
    CO-->>SO: candidate set + scores
    SO->>HM: filter by health & availability
    HM-->>SO: healthy candidates
    SO->>PF: create provider(model, provider, telemetry hooks)
    PF-->>SO: provider instance
    SO->>AP: execute request
    AP-->>SO: result / stream
    SO-->>NL: routed result
    NL-->>Client: response/stream

    note over HM,PF: Health metrics collected in real-time
    SO->>HM: update metrics (success/error/latency)
    HM-->>SO: updated health summary

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60–90 minutes

Poem

I hop between models, ears tuned to the air,
Weighing cost against speed with meticulous care.
A compass of health and a map of intent—
I route through the meadow, intelligently bent.
With twitches and ticks, I orchestrate fate,
Delivering answers—swift, precise, first-rate.

✨ Finishing Touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.