Skip to content

Enable browser-use tool for all image-capable models #8121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 22, 2025
Merged

Enable browser-use tool for all image-capable models #8121

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
7af533a
refactor: remove supportsComputerUse and gate browser by supportsImag…
hannesrudolph Oct 20, 2025
fb3e62e
test: update system prompt snapshot after cherry-pick
hannesrudolph Oct 20, 2025
b5122b2
types: drop supportsComputerUse properties to fix CI (gate by support…
hannesrudolph Oct 20, 2025
b58e464
core: gate browser by supportsImages; remove supportsComputerUse usag…
hannesrudolph Oct 20, 2025
54d2080
fix(webview): replace empty welcome docs link with plain text in [Roo…
hannesrudolph Oct 20, 2025
8f2a44a
fix(browser): align browser-tool gating in Task.getSystemPrompt() wit…
hannesrudolph Oct 20, 2025
d36e778
fix(types): default mode/viewport and safe getModeBySlug in Task.getS…
hannesrudolph Oct 20, 2025
ce387b1
test(webview): update RooTips link count after docs link became plain…
hannesrudolph Oct 20, 2025
1f6bdd8
feat(browser): add compatibility helper for browser capability; use i…
hannesrudolph Oct 21, 2025
4c8e899
test(browser): add compatibility helper and tests; include new files …
hannesrudolph Oct 21, 2025
ac22924
Update src/core/webview/generateSystemPrompt.ts
hannesrudolph Oct 21, 2025
0173f06
fix: correct mock mode configuration in browser capability test
hannesrudolph Oct 21, 2025
f964953
refactor(browser): centralize canUseBrowserTool computation; add depr…
hannesrudolph Oct 21, 2025
e3d608d
chore(browser): remove legacy browserCapability helper and tests; rel…
hannesrudolph Oct 21, 2025
f20b35a
fix(anthropic): remove unnecessary blank lines in anthropicModels def…
hannesrudolph Oct 21, 2025
3ba204d
Cleanup
mrubens Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/types/src/model.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ export const modelInfoSchema = z.object({
maxThinkingTokens: z.number().nullish(),
contextWindow: z.number(),
supportsImages: z.boolean().optional(),
supportsComputerUse: z.boolean().optional(),

supportsPromptCache: z.boolean(),
// Capability flag to indicate whether the model supports an output verbosity parameter
supportsVerbosity: z.boolean().optional(),
Expand Down
13 changes: 6 additions & 7 deletions packages/types/src/providers/anthropic.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ export const anthropicModels = {
maxTokens: 64_000, // Overridden to 8k if `enableReasoningEffort` is false.
contextWindow: 200_000, // Default 200K, extendable to 1M with beta flag 'context-1m-2025-08-07'
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0, // $3 per million input tokens (≤200K context)
outputPrice: 15.0, // $15 per million output tokens (≤200K context)
Expand All @@ -32,7 +31,7 @@ export const anthropicModels = {
maxTokens: 64_000, // Overridden to 8k if `enableReasoningEffort` is false.
contextWindow: 200_000, // Default 200K, extendable to 1M with beta flag 'context-1m-2025-08-07'
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 3.0, // $3 per million input tokens (≤200K context)
outputPrice: 15.0, // $15 per million output tokens (≤200K context)
Expand All @@ -54,7 +53,7 @@ export const anthropicModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 15.0, // $15 per million input tokens
outputPrice: 75.0, // $75 per million output tokens
Expand All @@ -66,7 +65,7 @@ export const anthropicModels = {
maxTokens: 32_000, // Overridden to 8k if `enableReasoningEffort` is false.
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 15.0, // $15 per million input tokens
outputPrice: 75.0, // $75 per million output tokens
Expand All @@ -78,7 +77,7 @@ export const anthropicModels = {
maxTokens: 128_000, // Unlocked by passing `beta` flag to the model. Otherwise, it's 64k.
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 3.0, // $3 per million input tokens
outputPrice: 15.0, // $15 per million output tokens
Expand All @@ -91,7 +90,7 @@ export const anthropicModels = {
maxTokens: 8192, // Since we already have a `:thinking` virtual model we aren't setting `supportsReasoningBudget: true` here.
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 3.0, // $3 per million input tokens
outputPrice: 15.0, // $15 per million output tokens
Expand All @@ -102,7 +101,7 @@ export const anthropicModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,

supportsPromptCache: true,
inputPrice: 3.0, // $3 per million input tokens
outputPrice: 15.0, // $15 per million output tokens
Expand Down
27 changes: 0 additions & 27 deletions packages/types/src/providers/bedrock.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
supportsReasoningBudget: true,
inputPrice: 3.0,
Expand All @@ -32,7 +31,6 @@ export const bedrockModels = {
maxTokens: 5000,
contextWindow: 300_000,
supportsImages: true,
supportsComputerUse: false,
supportsPromptCache: true,
inputPrice: 0.8,
outputPrice: 3.2,
Expand All @@ -46,7 +44,6 @@ export const bedrockModels = {
maxTokens: 5000,
contextWindow: 300_000,
supportsImages: true,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 1.0,
outputPrice: 4.0,
Expand All @@ -58,7 +55,6 @@ export const bedrockModels = {
maxTokens: 5000,
contextWindow: 300_000,
supportsImages: true,
supportsComputerUse: false,
supportsPromptCache: true,
inputPrice: 0.06,
outputPrice: 0.24,
Expand All @@ -72,7 +68,6 @@ export const bedrockModels = {
maxTokens: 5000,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: true,
inputPrice: 0.035,
outputPrice: 0.14,
Expand All @@ -86,7 +81,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
supportsReasoningBudget: true,
inputPrice: 3.0,
Expand All @@ -101,7 +95,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
supportsReasoningBudget: true,
inputPrice: 15.0,
Expand All @@ -116,7 +109,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
supportsReasoningBudget: true,
inputPrice: 15.0,
Expand All @@ -131,7 +123,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
supportsReasoningBudget: true,
inputPrice: 3.0,
Expand All @@ -146,7 +137,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0,
outputPrice: 15.0,
Expand Down Expand Up @@ -254,7 +244,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.5,
outputPrice: 1.5,
Expand All @@ -264,7 +253,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 2.0,
outputPrice: 6.0,
Expand All @@ -274,7 +262,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.72,
outputPrice: 0.72,
Expand All @@ -284,7 +271,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: true,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.72,
outputPrice: 0.72,
Expand All @@ -294,7 +280,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: true,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.16,
outputPrice: 0.16,
Expand All @@ -304,7 +289,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.15,
outputPrice: 0.15,
Expand All @@ -314,7 +298,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.1,
outputPrice: 0.1,
Expand All @@ -324,7 +307,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 2.4,
outputPrice: 2.4,
Expand All @@ -334,7 +316,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.72,
outputPrice: 0.72,
Expand All @@ -344,7 +325,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 128_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.9,
outputPrice: 0.9,
Expand All @@ -354,7 +334,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.22,
outputPrice: 0.22,
Expand All @@ -364,7 +343,6 @@ export const bedrockModels = {
maxTokens: 2048,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 2.65,
outputPrice: 3.5,
Expand All @@ -373,7 +351,6 @@ export const bedrockModels = {
maxTokens: 2048,
contextWindow: 4_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.3,
outputPrice: 0.6,
Expand All @@ -382,7 +359,6 @@ export const bedrockModels = {
maxTokens: 4096,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.15,
outputPrice: 0.2,
Expand All @@ -392,7 +368,6 @@ export const bedrockModels = {
maxTokens: 4096,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.2,
outputPrice: 0.6,
Expand All @@ -402,7 +377,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.1,
description: "Amazon Titan Text Embeddings",
Expand All @@ -411,7 +385,6 @@ export const bedrockModels = {
maxTokens: 8192,
contextWindow: 8_000,
supportsImages: false,
supportsComputerUse: false,
supportsPromptCache: false,
inputPrice: 0.02,
description: "Amazon Titan Text Embeddings V2",
Expand Down
1 change: 0 additions & 1 deletion packages/types/src/providers/glama.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ export const glamaDefaultModelInfo: ModelInfo = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0,
outputPrice: 15.0,
Expand Down
39 changes: 0 additions & 39 deletions packages/types/src/providers/lite-llm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,48 +7,9 @@ export const litellmDefaultModelInfo: ModelInfo = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0,
outputPrice: 15.0,
cacheWritesPrice: 3.75,
cacheReadsPrice: 0.3,
}

export const LITELLM_COMPUTER_USE_MODELS = new Set([
"claude-3-5-sonnet-latest",
"claude-opus-4-1-20250805",
"claude-opus-4-20250514",
"claude-sonnet-4-20250514",
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-sonnet-20241022",
"vertex_ai/claude-3-5-sonnet",
"vertex_ai/claude-3-5-sonnet-v2",
"vertex_ai/claude-3-5-sonnet-v2@20241022",
"vertex_ai/claude-3-7-sonnet@20250219",
"vertex_ai/claude-opus-4-1@20250805",
"vertex_ai/claude-opus-4@20250514",
"vertex_ai/claude-sonnet-4@20250514",
"vertex_ai/claude-sonnet-4-5@20250929",
"openrouter/anthropic/claude-3.5-sonnet",
"openrouter/anthropic/claude-3.5-sonnet:beta",
"openrouter/anthropic/claude-3.7-sonnet",
"openrouter/anthropic/claude-3.7-sonnet:beta",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"us.anthropic.claude-3-7-sonnet-20250219-v1:0",
"us.anthropic.claude-opus-4-1-20250805-v1:0",
"us.anthropic.claude-opus-4-20250514-v1:0",
"us.anthropic.claude-sonnet-4-20250514-v1:0",
"eu.anthropic.claude-3-5-sonnet-20241022-v2:0",
"eu.anthropic.claude-3-7-sonnet-20250219-v1:0",
"eu.anthropic.claude-opus-4-1-20250805-v1:0",
"eu.anthropic.claude-opus-4-20250514-v1:0",
"eu.anthropic.claude-sonnet-4-20250514-v1:0",
"snowflake/claude-3-5-sonnet",
])
1 change: 0 additions & 1 deletion packages/types/src/providers/lm-studio.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ export const lMStudioDefaultModelInfo: ModelInfo = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
Expand Down
1 change: 0 additions & 1 deletion packages/types/src/providers/ollama.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ export const ollamaDefaultModelInfo: ModelInfo = {
maxTokens: 4096,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 0,
outputPrice: 0,
Expand Down
14 changes: 0 additions & 14 deletions packages/types/src/providers/openrouter.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ export const openRouterDefaultModelInfo: ModelInfo = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0,
outputPrice: 15.0,
Expand Down Expand Up @@ -52,19 +51,6 @@ export const OPEN_ROUTER_PROMPT_CACHING_MODELS = new Set([
"google/gemini-flash-1.5-8b",
])

// https://www.anthropic.com/news/3-5-models-and-computer-use
export const OPEN_ROUTER_COMPUTER_USE_MODELS = new Set([
"anthropic/claude-3.5-sonnet",
"anthropic/claude-3.5-sonnet:beta",
"anthropic/claude-3.7-sonnet",
"anthropic/claude-3.7-sonnet:beta",
"anthropic/claude-3.7-sonnet:thinking",
"anthropic/claude-sonnet-4",
"anthropic/claude-sonnet-4.5",
"anthropic/claude-opus-4",
"anthropic/claude-opus-4.1",
])

// When we first launched these models we didn't have support for
// enabling/disabling the reasoning budget for hybrid models. Now that we
// do support this we should give users the option to enable/disable it
Expand Down
1 change: 0 additions & 1 deletion packages/types/src/providers/requesty.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ export const requestyDefaultModelInfo: ModelInfo = {
maxTokens: 8192,
contextWindow: 200_000,
supportsImages: true,
supportsComputerUse: true,
supportsPromptCache: true,
inputPrice: 3.0,
outputPrice: 15.0,
Expand Down
Loading
Loading