Skip to content

feat: update Groq models for better tool calling support #3676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DOsinga merged 3 commits into from
Jul 29, 2025
Merged

feat: update Groq models for better tool calling support #3676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
35a3627
feat: update Groq models for better tool calling support
Jul 27, 2025
a3a29b2
feat: add token limits for Groq models
Jul 28, 2025
7b64d4c
Merge branch 'main' into update-groq-models
DOsinga Jul 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion crates/goose/src/model.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,14 @@ static MODEL_SPECIFIC_LIMITS: Lazy<HashMap<&'static str, usize>> = Lazy::new(||
map.insert("llama3.3", 128_000);

// x.ai Grok models, https://docs.x.ai/docs/overview
map.insert("grok", 131_072);

// Groq models, https://console.groq.com/docs/models
map.insert("gemma2-9b", 8_192);
map.insert("kimi-k2", 131_072);
map.insert("qwen3-32b", 131_072);
map.insert("grok-3", 131_072);
map.insert("grok-4", 256_000); // 256K

map.insert("qwen3-coder", 262_144); // 262K

map
Expand Down
9 changes: 7 additions & 2 deletions crates/goose/src/providers/groq.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,13 @@ use std::time::Duration;
use url::Url;

pub const GROQ_API_HOST: &str = "https://api.groq.com";
pub const GROQ_DEFAULT_MODEL: &str = "llama-3.3-70b-versatile";
pub const GROQ_KNOWN_MODELS: &[&str] = &["gemma2-9b-it", "llama-3.3-70b-versatile"];
pub const GROQ_DEFAULT_MODEL: &str = "moonshotai/kimi-k2-instruct";
pub const GROQ_KNOWN_MODELS: &[&str] = &[
"gemma2-9b-it",
"llama-3.3-70b-versatile",
"moonshotai/kimi-k2-instruct",
"qwen/qwen3-32b",
];
Copy link
Collaborator

@DOsinga DOsinga Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure these models also have their token limits defined in models.rs


pub const GROQ_DOC_URL: &str = "https://console.groq.com/docs/models";

Expand Down
35 changes: 35 additions & 0 deletions documentation/docs/getting-started/providers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,41 @@ These free options are a great way to get started with Goose and explore its cap
:::


### Groq
Groq provides free access to open source models with high-speed inference. To use Groq with Goose, you need an API key from [Groq Console](https://console.groq.com/keys).

Groq offers several open source models that support tool calling:
- **moonshotai/kimi-k2-instruct** - Mixture-of-Experts model with 1 trillion parameters, optimized for agentic intelligence and tool use
- **qwen/qwen3-32b** - 32.8 billion parameter model with advanced reasoning and multilingual capabilities
- **gemma2-9b-it** - Google's Gemma 2 model with instruction tuning
- **llama-3.3-70b-versatile** - Meta's Llama 3.3 model for versatile applications

To set up Groq with Goose, follow these steps:

<Tabs groupId="interface">
<TabItem value="ui" label="Goose Desktop" default>
**To update your LLM provider and API key:**

1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar.
2. Click the `Settings` button on the sidebar.
3. Click the `Models` tab.
4. Click `Configure Providers`
5. Choose `Groq` as provider from the list.
6. Click `Configure`, enter your API key, and click `Submit`.

</TabItem>
<TabItem value="cli" label="Goose CLI">
1. Run:
```sh
goose configure
```
2. Select `Configure Providers` from the menu.
3. Follow the prompts to choose `Groq` as the provider.
4. Enter your API key when prompted.
5. Enter the Groq model of your choice (e.g., `moonshotai/kimi-k2-instruct`).
</TabItem>
</Tabs>

### Google Gemini
Google Gemini provides a free tier. To start using the Gemini API with Goose, you need an API Key from [Google AI studio](https://aistudio.google.com/app/apikey).

Expand Down
Loading