feat: Refactor OpenRouter provider to use Vercel AI SDK #10778
Conversation
dosubot
bot
added
size:XXL
Review of commit ae9636e. No new issues found. 2 of 5 original issues remain unresolved.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| private client: OpenAI | ||
| protected models: ModelRecord = {} | ||
| protected endpoints: ModelRecord = {} | ||
| private readonly providerName = "OpenRouter" |
There was a problem hiding this comment.
The old implementation had a getReasoningDetails() method and currentReasoningDetails accumulator that Task.ts relies on (see line 959 in Task.ts). This method accumulates reasoning_details from the stream (handling reasoning.text, reasoning.summary, and reasoning.encrypted types) and provides them for persistence in API conversation history. This is required for models like Gemini 3 via OpenRouter that use the reasoning.encrypted format - without it, reasoning context won't be preserved across multi-turn conversations, potentially breaking those model integrations.
Fix it with Roo Code or mention @roomote and request a fix.
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
5260473 to
d062770
Compare
There was a problem hiding this comment.
Additional Issues Found in Re-review
The previously flagged issue with missing getReasoningDetails() method remains unresolved. Additionally, I've identified a few more regressions in this refactor:
1. Telemetry Loss
Error handling no longer reports exceptions to telemetry.
2. Usage Data Loss
Detailed usage information (cost, cache tokens, reasoning tokens) is no longer yielded.
3. Model-Specific Handling Removed
Special handling for DeepSeek R1, Gemini, and Anthropic models has been removed.
See inline comments for details.
| } | ||
| // After streaming completes, yield usage information | ||
| const usage = await result.usage | ||
| const totalUsage = await result.totalUsage |
There was a problem hiding this comment.
Telemetry Loss: The previous implementation called TelemetryService.instance.captureException(apiError) to report errors. This new error handling loses telemetry tracking, which is important for monitoring production errors.
Consider adding telemetry capture:
} catch (error) {
const errorMessage = error instanceof Error ? error.message : String(error)
const apiError = new ApiProviderError(errorMessage, this.providerName, modelId, 'createMessage')
TelemetryService.instance.captureException(apiError)
yield {
type: 'error',
error: 'OpenRouterError',
message: `${this.providerName} API Error: ${errorMessage}`,
}
}| tools, | ||
| toolChoice: metadata?.tool_choice as any, | ||
| providerOptions, | ||
| }) |
There was a problem hiding this comment.
Usage Data Regression: The previous implementation yielded detailed usage information including:
totalCostfromcost_details.upstream_inference_cost + costcacheReadTokensfromprompt_tokens_details.cached_tokensreasoningTokensfromcompletion_tokens_details.reasoning_tokens
The new implementation only yields basic inputTokens and outputTokens. This loses cost tracking and token detail information that may be important for usage analytics and cost monitoring.
The AI SDK's result.usage may have different properties - please verify what data is available and restore as much usage detail as possible.
| @@ -212,323 +83,59 @@ export class OpenRouterHandler extends BaseProvider implements SingleCompletionH | |||
| metadata?: ApiHandlerCreateMessageMetadata, | |||
There was a problem hiding this comment.
Model-Specific Handling Removed: The previous implementation had important model-specific handling that has been removed:
- DeepSeek R1: Converted messages using
convertToR1Format()which uses user role instead of system role - Gemini Models: Had
sanitizeGeminiMessages()and fake encrypted block injection for preserving reasoning context when switching models - Anthropic Models: Added
x-anthropic-beta: fine-grained-tool-streaming-2025-05-14header - Prompt Caching: Called
addGeminiCacheBreakpoints()oraddAnthropicCacheBreakpoints()for supported models - Reasoning Parameter: The
reasoningparameter (including exclude logic for Gemini 2.5 Pro) is no longer passed to the API
These may cause regressions for specific model providers. Please verify if the AI SDK handles these cases automatically, or if they need to be re-implemented.
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
- Add @openrouter/ai-sdk-provider and ai packages to devDependencies - Align zod versions across monorepo to prevent type issues - Add src/api/transform/ai-sdk.ts with: - convertToAiSdkMessages(): Anthropic messages to CoreMessage format - convertToolsForAiSdk(): OpenAI tools to AI SDK tool format - processAiSdkStreamPart(): AI SDK stream events to ApiStreamChunk - Add comprehensive tests in src/api/transform/__tests__/ai-sdk.spec.ts
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
d8f0c64 to
11fd793
Compare
Review of commit 11fd793. No new issues found. 2 of 5 original issues remain unresolved.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
- Fix version regression: restore version to 3.45.0 (was incorrectly 3.43.0) - Handle URL source type in convertToAiSdkMessages for images - Add test for URL image source handling
- Replace direct OpenAI SDK usage with @openrouter/ai-sdk-provider - Add 'ai' package for streamText() and generateText() functions - Extract reusable AI SDK conversion utilities to src/api/transform/ai-sdk.ts: - convertToAiSdkMessages(): Anthropic messages to CoreMessage format - convertToolsForAiSdk(): OpenAI tools to AI SDK tool format - processAiSdkStreamPart(): AI SDK stream events to ApiStreamChunk - Update tests to mock AI SDK functions instead of OpenAI client - Add comprehensive tests for the new ai-sdk.ts utility module This follows Vercel's AI SDK provider pattern for standardized LLM integration and enables easier migration of other providers in the future.
- Fixed tool result output format to use typed object { type: 'text', value: string }
instead of plain string to satisfy AI SDK validation schema
- Added tool name resolution by building a map of tool call IDs to names
- Updated tests to reflect new output format
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
- Add support for 'reasoning' and 'text' event types in AI SDK stream processing - Pass reasoning parameters via createOpenRouter extraBody instead of providerOptions - Support both effort-based (effort: 'high') and budget-based (max_tokens: N) reasoning - Add comprehensive tests for reasoning parameter passing and stream event handling - Fixes reasoning tokens not being displayed for models like DeepSeek R1 and Gemini Thinking Changes: - src/api/transform/ai-sdk.ts: Add 'text' and 'reasoning' event type handlers - src/api/providers/openrouter.ts: Pass reasoning via extraBody in provider creation - Add tests for new event types and reasoning parameter flow - All 53 tests passing
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
11fd793 to
ae9636e
Compare
Summary
Migrate the OpenRouter provider from the OpenAI SDK to the Vercel AI SDK for a more standardized and maintainable implementation.
Closes COM-502 (part 2/2)
Problem
The current OpenRouter provider implementation uses the raw OpenAI SDK directly, which requires complex message transformation and reasoning token handling code that is difficult to maintain.
Solution
Refactor the OpenRouter provider to use the
@openrouter/ai-sdk-providerpackage which integrates with Vercel's AI SDK. This provides:streamTextandgenerateTextCoreMessageformatChanges
src/api/providers/openrouter.tsto use AI SDK patternssrc/api/providers/__tests__/openrouter.spec.tswith new test patternsTesting
tsc --noEmitpasses in bothpackages/typesandsrcRelated
Important
Refactor
OpenRouterHandlerto use Vercel AI SDK, simplifying implementation and improving maintainability with updated tests.OpenRouterHandlerinopenrouter.tsto use Vercel AI SDK, replacing direct OpenAI SDK usage.createOpenRouterProvider()to initialize provider with reasoning parameters.normalizeUsage()to extract detailed usage metrics from AI SDK response.createMessage()andcompletePrompt()to use AI SDK'sstreamTextandgenerateText.openrouter.spec.tsto mock AI SDK functions and test new behavior.This description was created by
for ae9636e. You can customize this summary. It will automatically update as commits are pushed.