Add AI provider configurations for local and cloud models - #248
Conversation
|
Warning Rate limit exceeded@shunkakinoki has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 21 minutes and 34 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds provider configuration to Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant Opencode as Opencode Module
participant Router as Provider Router
participant LM as lmstudio
participant OL as ollama
participant OR as openrouter
participant BaseTen as baseten (via openrouter)
User->>Opencode: Request(model, prompt)
Opencode->>Router: Select provider per config
alt Models mapped to lmstudio
Router->>LM: Invoke with baseURL
LM-->>Opencode: Response
else Models mapped to ollama
Router->>OL: Invoke with baseURL
OL-->>Opencode: Response
else Routed via openrouter
Router->>OR: Invoke (allow_fallbacks=false)
OR->>BaseTen: Provider(order[0])
BaseTen-->>OR: Response
OR-->>Opencode: Response
end
Opencode-->>User: Result
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🧪 Early access (Sonnet 4.5): enabledWe are currently testing the Sonnet 4.5 model, which is expected to improve code review quality. However, this model may lead to increased noise levels in the review comments. Please disable the early access features if the noise level causes any inconvenience. Note:
Comment |
Summary of ChangesHello @shunkakinoki, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the AI model provider options within the 'opencode' system by introducing configurations for local models via LM Studio and Ollama, alongside a refinement of the existing OpenRouter cloud provider. The changes aim to offer greater flexibility and control over which AI models can be utilized, catering to both local development and specific cloud-based model requirements. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds configurations for several AI providers, including local providers like LM Studio and Ollama, and enhances the OpenRouter configuration. The changes are well-structured. I've identified a couple of areas for improvement: one is a potential logical error in the OpenRouter configuration where allow_fallbacks is set to false despite the PR description suggesting otherwise, and the other is a minor inconsistency in the baseURL for local providers. Addressing these points will improve the correctness and consistency of the configuration.
| "order": [ | ||
| "baseten" | ||
| ], | ||
| "allow_fallbacks": false |
There was a problem hiding this comment.
The pull request description states that this change enhances the OpenRouter provider with a fallback configuration. However, allow_fallbacks is set to false, which explicitly disables this feature. If the intent is to allow OpenRouter to use other providers for the kimi-k2 model when baseten is unavailable, this value should be set to true.
"allow_fallbacks": true
| "npm": "@ai-sdk/openai-compatible", | ||
| "name": "LM Studio (local)", | ||
| "options": { | ||
| "baseURL": "http://127.0.0.1:1234/v1" |
There was a problem hiding this comment.
For consistency with the ollama provider configuration (line 23), which uses localhost, consider using localhost here as well instead of 127.0.0.1. While both are functionally similar on most systems, using a consistent hostname for local services improves the readability and maintainability of the configuration file.
"baseURL": "http://localhost:1234/v1"
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
home-manager/modules/opencode/opencode.jsonc(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
home-manager/**
📄 CodeRabbit inference engine (.cursor/rules/general.mdc)
Keep home-manager configurations under home-manager/
Files:
home-manager/modules/opencode/opencode.jsonc
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: e2e-run (NixOS, ubuntu-latest)
- GitHub Check: e2e-run (Ubuntu, ubuntu-latest)
- GitHub Check: e2e-run (MacOS, macos-latest)
- GitHub Check: docker-build-push (linux/amd64, -amd64, amd64)
- GitHub Check: nix-nixos
- GitHub Check: nix-darwin
- GitHub Check: nix-linux
🔇 Additional comments (4)
home-manager/modules/opencode/opencode.jsonc (4)
5-5: Verify autoupdate behavior and implications.The
autoupdateflag is enabled but it's unclear what will be automatically updated (configuration, providers, models, or the application itself). Automatic updates could introduce breaking changes or unexpected behavior.Consider documenting what this flag controls or verify that automatic updates align with your stability requirements.
7-18: LM Studio provider configuration looks good.The local provider configuration is properly structured with appropriate localhost URL and model mapping. Ensure that LM Studio is running and accessible at
http://127.0.0.1:1234/v1when this configuration is used.
31-44: OpenRouter provider configuration differs from local providers.The openrouter provider configuration lacks
npmandoptions.baseURLfields present in the lmstudio and ollama providers. This is likely intentional if openrouter is a built-in provider, but please verify this is the expected structure.
34-40: Fallback disabled may impact reliability.The
allow_fallbacks: falsesetting for the kimi-k2 model means if the "baseten" provider is unavailable, the request will fail rather than trying alternative providers. This could reduce reliability.Consider whether this is the desired behavior or if allowing fallbacks would provide better resilience.
| "ollama": { | ||
| "npm": "@ai-sdk/openai-compatible", | ||
| "name": "Ollama (local)", | ||
| "options": { | ||
| "baseURL": "http://localhost:11434/v1" | ||
| }, | ||
| "models": { | ||
| "openai/gpt-oss-20b": { | ||
| "name": "OpenAI: gpt-oss-20b" | ||
| } | ||
| } | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
Ollama provider configuration is correct, but verify model naming.
The Ollama configuration is properly structured. However, the model identifier "openai/gpt-oss-20b" with display name "OpenAI: gpt-oss-20b" may be confusing since "gpt-oss-20b" doesn't appear to be an official OpenAI model. Verify that:
- The model identifier matches what Ollama expects
- The "OpenAI:" prefix in the display name accurately represents the model's origin
🏁 Script executed:
#!/bin/bash
# Description: Search for documentation or references to gpt-oss-20b model in the codebase
# to verify if this is an established naming convention
rg -i "gpt-oss-20b" -C 3Length of output: 924
🌐 Web query:
Does Ollama support a model named "gpt-oss-20b" and what is its correct model identifier?
💡 Result:
Yes. Ollama supports OpenAI's 20B gpt-oss model. The correct Ollama model identifier is gpt-oss:20b (often shown as "gpt-oss-20b" in prose, but use gpt-oss:20b in the Ollama CLI). [1][2]
Sources:
[1] Ollama model page for gpt-oss:20b.
[2] Ollama blog / docs showing usage (examples: ollama run gpt-oss:20b).
Use the proper Ollama model naming
- In home-manager/modules/opencode/opencode.jsonc (lines 19–30), change the model key from
"openai/gpt-oss-20b"
to
"gpt-oss:20b". - Update the display name from
"OpenAI: gpt-oss-20b"
to something like
"Ollama: gpt-oss:20b".
🤖 Prompt for AI Agents
In home-manager/modules/opencode/opencode.jsonc around lines 19 to 30 the Ollama
model is using the incorrect key and display name; change the model key from
"openai/gpt-oss-20b" to "gpt-oss:20b" and update the display name from "OpenAI:
gpt-oss-20b" to a vendor-appropriate label such as "Ollama: gpt-oss:20b" so the
configuration uses the proper Ollama model identifier and clear naming.
1 participant
Summary