feat(models): add Bedrock Llama models - #1007
Conversation
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
❌ Preview Environment deleted from BunnyshellAvailable commands (reply to this comment):
|
WalkthroughAdds an aws-bedrock provider and pricing entries for multiple Llama models, introduces a new model "llama-3.1-70b-instruct", and updates capability flags (vision, streaming, tools), contextSize, and maxOutput for several existing models in the meta model registry. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (2)**/*.{ts,tsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (AGENTS.md)
Files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
🔇 Additional comments (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Mark Llama 3, 3.2, and 3.3 models as test: "skip" since they use INFERENCE_PROFILE instead of ON_DEMAND and require additional setup. Only Llama 3.1 8B and 70B models are tested and working. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/models/src/models/meta.ts(6 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: This is a pure TypeScript project—do not useanyoras anyunless absolutely necessary
Always use top-level ES imports; never userequireor dynamicimport()
Files:
packages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic import()
Files:
packages/models/src/models/meta.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: lint / run
- GitHub Check: generate / run
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (1)
- GitHub Check: autofix
🔇 Additional comments (5)
packages/models/src/models/meta.ts (5)
107-128: LGTM with note on provider coverage.The AWS Bedrock provider configuration looks correct. Note that other providers (Nebius, Together.ai, etc.) that support Llama 3.1 70B could be added in future PRs if needed.
413-414: Pricing and vision support verified: input $0.24/output $0.97 per 1M tokens matches AWS Bedrock rates; model supports native text + image inputs.
283-295: Confirm region-specific AWS Bedrock pricing
- AWS Bedrock does not publish a flat 0.72 $/1 M-token rate; verify the exact input/output token rates for your target region.
tools: falseis correct—Bedrock’s Llama 3.3 70B Instruct does not support external tools.
396-396: Vision support confirmed for Scout 17B.AWS Bedrock lists
meta.llama4-scout-17b-instruct-v1:0as a text + image model, sovision: trueis correct.
182-194: Vision support and pricing confirmed
AWS Bedrock’s meta.llama3-2-11b-instruct-v1:0 supports vision, and the model’s pricing of $0.35 per 1,000 input and output tokens matches the values set.
| { | ||
| id: "llama-3-8b-instruct", | ||
| name: "Llama 3 8B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| test: "skip", | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-8b-instruct-v1:0", | ||
| inputPrice: 0.3 / 1e6, | ||
| outputPrice: 0.6 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 8192, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, | ||
| { | ||
| id: "llama-3-70b-instruct", | ||
| name: "Llama 3 70B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| test: "skip", | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-70b-instruct-v1:0", | ||
| inputPrice: 2.65 / 1e6, | ||
| outputPrice: 3.5 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 8192, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
Verify AWS Bedrock pricing and specifications for Llama 3 models.
Ensure that the pricing, context size, and maxOutput values match the official AWS Bedrock documentation for these models. The test: "skip" flag suggests these models may need verification before testing is enabled.
🌐 Web query:
What are the official AWS Bedrock pricing rates and specifications (context size, max output tokens) for meta.llama3-8b-instruct-v1:0 and meta.llama3-70b-instruct-v1:0?
💡 Result:
Short answer — current published rates and specs (sources below):
-
meta.llama3-8b-instruct-v1:0
- Context length: 128,000 tokens. [1]
- Price (on-demand): $0.0003 per 1,000 input tokens; $0.0003 per 1,000 output tokens (equivalently $0.3 / $0.3 per 1M tokens). [1][2]
- Practical max generated output observed on Bedrock: commonly capped at ~8,192 tokens (see notes). [3]
-
meta.llama3-70b-instruct-v1:0
- Context length: 128,000 tokens. [1]
- Price (on-demand): $0.00265 per 1,000 input tokens; $0.0035 per 1,000 output tokens (equivalently $2.65 / $3.50 per 1M tokens). [1][2]
- Practical max generated output observed on Bedrock: commonly capped at ~8,192 tokens (see notes). [3]
Notes:
- AWS Bedrock official pricing page describes Bedrock billing and model pricing framework but does not list every model-rate table in a single page; third‑party Bedrock model price tables (linked below) report the per‑model token rates above. [4][1]
- Several Bedrock users/reporting threads indicate that although some Llama‑3 family variants support long context windows (e.g., 128K), Bedrock’s response/generation output is often limited in practice to ~8,192 tokens for many Llama model versions; verify by testing the specific model/version in your target AWS region/account. [3]
Sources:
[1] Bedrock model price/spec table (aggregated listing) — meta.llama3-* entries.
[2] Third‑party Bedrock model pricing summary (per‑1M / per‑1k mapping).
[3] AWS re:Post thread and community reports re: Bedrock Llama token/output limits.
[4] AWS Bedrock pricing and billing documentation (Amazon).
Fix llama-3-8b-instruct spec
- In packages/models/src/models/meta.ts (lines 4–49): set
contextSizeto 128000. - Change
outputPricefrom 0.6/1e6 to 0.3/1e6.
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 4 to 49, update the Llama 3
8B Instruct provider entry: set contextSize from 8192 to 128000 and change
outputPrice from 0.6 / 1e6 to 0.3 / 1e6 in the aws-bedrock provider object for
id "llama-3-8b-instruct".
| { | ||
| id: "llama-3.2-1b-instruct", | ||
| name: "Llama 3.2 1B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| test: "skip", | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-2-1b-instruct-v1:0", | ||
| inputPrice: 0.1 / 1e6, | ||
| outputPrice: 0.1 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, | ||
| { | ||
| id: "llama-3.2-3b-instruct", | ||
| name: "Llama 3.2 3B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| test: "skip", | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-2-3b-instruct-v1:0", | ||
| inputPrice: 0.15 / 1e6, | ||
| outputPrice: 0.15 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
Verify pricing for Llama 3.2 1B and 3B models.
Please confirm the pricing rates for these smaller models match AWS Bedrock documentation. The symmetric pricing and test: "skip" flags suggest these are newly added models pending verification.
🌐 Web query:
What are the official AWS Bedrock pricing rates for meta.llama3-2-1b-instruct-v1:0 and meta.llama3-2-3b-instruct-v1:0?
💡 Result:
AWS’s Bedrock pricing page does not list per-model rates for those Llama 3.2 instruct variants. The public/third‑party sources that track Bedrock prices report:
- meta.llama3-2-1b-instruct-v1:0 — $0.10 per 1,000 input tokens and $0.10 per 1,000 output tokens. [AWS pricing page (general Llama 3.2 section) [1]; price listing [2][3]]
- meta.llama3-2-3b-instruct-v1:0 — $0.15 per 1,000 input tokens and $0.15 per 1,000 output tokens. [AWS pricing page (general Llama 3.2 section) [1]; price listing [2][4]]
Sources:
[1] AWS Bedrock pricing (Llama 3 section). [2] ai-pricing.info model list for Bedrock. [3] tokencostauto / PyPI listing. [4] litellm-mirror pricing JSON.
Correct pricing units for Llama 3.2 instruct models
The inputPrice/outputPrice values are off by three orders of magnitude. AWS lists $0.10/$0.15 per 1,000 tokens (equivalent to 100/1e6 and 150/1e6 per token), not 0.1/1e6 and 0.15/1e6.
File: packages/models/src/models/meta.ts (lines 129–174)
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 129 to 174, the
inputPrice/outputPrice for the two Llama 3.2 instruct models use 0.1/1e6 and
0.15/1e6 which are off by three orders of magnitude; AWS lists $0.10/$0.15 per
1,000 tokens (per-token = 0.0001/0.00015 = 100/1e6 and 150/1e6). Update the
inputPrice and outputPrice for "llama-3.2-1b-instruct" from 0.1/1e6 to 100/1e6
and for "llama-3.2-3b-instruct" from 0.15/1e6 to 150/1e6 (apply the same change
to both inputPrice and outputPrice fields).
| { | ||
| id: "llama-3.2-90b-instruct", | ||
| name: "Llama 3.2 90B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| test: "skip", | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-2-90b-instruct-v1:0", | ||
| inputPrice: 2.0 / 1e6, | ||
| outputPrice: 2.0 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: true, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
Verify vision support and pricing for Llama 3.2 90B.
This model has vision: true enabled per the PR objectives. Please confirm the vision capability and pricing (2.0/2.0 per 1M) against AWS Bedrock documentation.
🌐 Web query:
Does AWS Bedrock support vision capabilities for meta.llama3-2-90b-instruct-v1:0, and what is the official pricing?
💡 Result:
Short answer:
-
Yes — Amazon Bedrock’s Llama 3.2 90B (meta.llama3-2-90b-instruct-v1:0) supports vision (image reasoning / multimodal) in Bedrock. [AWS announcement, Mar 14, 2025]. [1]
-
Official pricing: consult the Amazon Bedrock pricing page for the authoritative, region‑specific rates. AWS documents Llama 3.2 under Bedrock pricing; third‑party price listings that reflect observed Bedrock on‑demand rates report about $2.00 per 1,000 input tokens and $2.00 per 1,000 output tokens for meta.llama3-2-90b. Treat those third‑party tables as observed rates and verify in the AWS Bedrock pricing console for your region before use. [2][3]
Sources:
- AWS “Meta’s Llama 3.2 models are now available …” (announces multimodal/vision support for Llama 3.2 11B and 90B). [AWS, Mar 14, 2025].
- Amazon Bedrock pricing page (model pricing sections — check region/console for definitive current rates).
- Third‑party price tables aggregating Bedrock model rates (examples showing $2.00/1k tokens for meta.llama3-2-90b).
Would you like me to open the Bedrock pricing console for your preferred AWS region and pull the current, official per‑token rates?
Update pricing for Llama 3.2 90B to match AWS Bedrock rates
Vision support is correct. AWS lists Llama 3.2 90B at $2.00 per 1 000 input/output tokens, so set inputPrice/outputPrice to 2.0 / 1e3 (not 2.0 / 1e6). Verify region-specific rates in the AWS console.
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 209 to 231, the
inputPrice/outputPrice for the "llama-3.2-90b-instruct" provider are incorrectly
set to 2.0 / 1e6; update both inputPrice and outputPrice to 2.0 / 1e3 to reflect
AWS Bedrock's $2.00 per 1,000 tokens rate, keep all other fields (vision true,
etc.) unchanged, and add a short comment noting the rate source or to verify
region-specific rates in AWS console.
Remove Llama 3, 3.2, and 3.3 Bedrock models that use INFERENCE_PROFILE instead of ON_DEMAND. Only keeping Llama 3.1 8B and 70B which are tested and working. Models can be added back when INFERENCE_PROFILE support is implemented. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/models/src/models/meta.ts (1)
11-22: Fix maxOutput to match AWS Bedrock spec (128000, not 2048).The
maxOutputis set to 2048, but AWS Bedrock supports 128,000 tokens for meta.llama3-1-8b-instruct-v1:0. This was previously flagged in an earlier review.Apply this diff:
{ providerId: "aws-bedrock", modelName: "meta.llama3-1-8b-instruct-v1:0", inputPrice: 0.22 / 1e6, outputPrice: 0.22 / 1e6, requestPrice: 0, contextSize: 128000, - maxOutput: 2048, + maxOutput: 128000, streaming: true, vision: false, tools: false, },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/models/src/models/meta.ts(4 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: This is a pure TypeScript project—do not useanyoras anyunless absolutely necessary
Always use top-level ES imports; never userequireor dynamicimport()
Files:
packages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic import()
Files:
packages/models/src/models/meta.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: lint / run
- GitHub Check: generate / run
- GitHub Check: test / run
- GitHub Check: build / run
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (4)
- GitHub Check: e2e-shards (3)
- GitHub Check: autofix
🔇 Additional comments (2)
packages/models/src/models/meta.ts (2)
272-273: Pricing and vision support updates confirmed.The updated rates (input: $0.24, output: $0.97 per 1M tokens) and multimodal capabilities (text + image support) align with AWS Bedrock specs for meta.llama4-maverick-17b-instruct-v1:0.
255-255: Enable vision support confirmed for Llama 4 Scout 17B
AWS Bedrock officially supports multimodal image/vision input for meta.llama4-scout-17b-instruct-v1:0, so settingvision: trueis correct.
| { | ||
| id: "llama-3.1-70b-instruct", | ||
| name: "Llama 3.1 70B Instruct", | ||
| family: "meta", | ||
| deprecatedAt: undefined, | ||
| deactivatedAt: undefined, | ||
| providers: [ | ||
| { | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-1-70b-instruct-v1:0", | ||
| inputPrice: 0.99 / 1e6, | ||
| outputPrice: 0.99 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, | ||
| ], | ||
| jsonOutput: false, | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
Verify pricing and maxOutput for Llama 3.1 70B on AWS Bedrock.
The maxOutput is set to 2048, but given that the smaller 8B variant supports 128,000 tokens and contextSize is already 128000, this seems inconsistent. Please verify both the pricing (0.99/1e6 for input and output) and the correct maxOutput specification.
🌐 Web query:
What are the official AWS Bedrock pricing rates and max output token limit for meta.llama3-1-70b-instruct-v1:0?
💡 Result:
Short answer:
-
Official AWS Bedrock model-page / pricing docs do not publish a per-model on‑demand token rate for meta.llama3-1-70b-instruct-v1:0 on the main pricing page; Bedrock’s pricing page shows Meta model families but not a single-line public rate for that exact model. [1]
-
Community/third‑party price tables that mirror Bedrock listings report: $0.99 per 1,000 input tokens and $0.99 per 1,000 output tokens for meta.llama3-1-70b-instruct-v1:0. (Use with caution — these are derived from mirrored tables, not an explicit single-row AWS price page.) [2]
-
Max output token limit: in many Bedrock regions the practical per‑call output limit observed for Llama models (including meta.llama3-1-70b-instruct-v1:0) is 8,192 tokens; however, Bedrock supports a 128K (128,000) context-version of Meta Llama 3.1 70B for Provisioned Throughput in specific regions (e.g., us-east-2 and us-west-2) where the model variant is listed as 128K — in that configuration the model’s input context can be up to 128K but documented per-call output limits on AWS pages and user reports show smaller output caps (commonly 2,048–8,192 depending on variant/region/provisioning). See AWS model/throughput support and community reports. [3][4]
Sources:
- AWS Bedrock pricing & Meta section. [1]
- Community/mirror tables showing per‑model token prices (meta.llama3-1-70b-instruct-v1:0 = $0.99/$0.99). [2]
- AWS docs / Provisioned Throughput supported models (lists meta.llama3-1-70b-instruct-v1:0:128k availability) and community re:Post discussion about 8,192 practical limit. [3][4]
If you want, I can: (a) open an AWS support/pricing link to request the official per‑region on‑demand rates, or (b) fetch the exact per‑region on‑demand/Provisioned‑Throughput prices for that model and confirm the precise per‑call output token cap for the region you care about.
🏁 Script executed:
#!/bin/bash
rg -n "llama-3.1-8b-instruct" -C3 packages/models/src/models/meta.tsLength of output: 356
Fix AWS Bedrock pricing and context/output limits for llama-3.1-70b-instruct
- inputPrice/outputPrice should be
0.99 / 1e3(≈ $0.00099/token), not0.99 / 1e6. - For the on-demand 8K context variant, set
contextSize: 8192andmaxOutputto the model’s per-call cap (e.g., 8192); only usecontextSize: 128000when using the 128K Provisioned-Throughput variant—and update modelName/pricing accordingly.
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 61 to 82, the AWS Bedrock
provider entry for "llama-3.1-70b-instruct" has incorrect pricing and
context/output limits; change inputPrice and outputPrice from 0.99 / 1e6 to 0.99
/ 1e3, and set contextSize to 8192 and maxOutput to 8192 for the on-demand 8K
context variant (only use contextSize: 128000 and different pricing/modelName
when representing the 128K provisioned-throughput variant), and ensure the
provider.modelName and pricing reflect the chosen variant consistently.
Update pricing for Llama 3.1 models based on official AWS pricing: - Llama 3.1 8B: $2.20 input / $6.60 output per 1M tokens - Llama 3.1 70B: $10.60 input / $31.80 output per 1M tokens Previous pricing was 10x lower than actual AWS rates. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Revert back to original pricing which was correct: - Llama 3.1 8B: $0.22 input / $0.22 output per 1M tokens - Llama 3.1 70B: $0.72 input / $0.72 output per 1M tokens AWS pricing is $0.00022/$0.00072 per 1K tokens, which converts to $0.22/$0.72 per 1M tokens. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/models/src/models/meta.ts (1)
61-82: Fix critical pricing and specification errors for aws-bedrock llama-3.1-70b-instruct.Multiple issues with this new model entry:
Pricing is incorrect: AWS charges $0.99 per 1,000 tokens (symmetric), but the code shows asymmetric pricing that's 100× too low for input and 30× too low for output.
Context/output limits mismatch: Past review indicates the on-demand variant uses 8K context with 8K max output, while the 128K variant requires Provisioned Throughput with different pricing.
Apply this diff to fix pricing and align with the on-demand 8K variant:
{ id: "llama-3.1-70b-instruct", name: "Llama 3.1 70B Instruct", family: "meta", deprecatedAt: undefined, deactivatedAt: undefined, providers: [ { providerId: "aws-bedrock", modelName: "meta.llama3-1-70b-instruct-v1:0", - inputPrice: 10.6 / 1e6, - outputPrice: 31.8 / 1e6, + inputPrice: 0.99 / 1e3, + outputPrice: 0.99 / 1e3, requestPrice: 0, - contextSize: 128000, - maxOutput: 2048, + contextSize: 8192, + maxOutput: 8192, streaming: true, vision: false, tools: false, }, ], jsonOutput: false, },If you need to support the 128K Provisioned Throughput variant instead, update the modelName and pricing accordingly.
Based on past review verification results.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/models/src/models/meta.ts(4 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: This is a pure TypeScript project—do not useanyoras anyunless absolutely necessary
Always use top-level ES imports; never userequireor dynamicimport()
Files:
packages/models/src/models/meta.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
Always use top-level import; never use require() or dynamic import()
Files:
packages/models/src/models/meta.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: build / run
- GitHub Check: test / run
- GitHub Check: generate / run
- GitHub Check: lint / run
- GitHub Check: autofix
- GitHub Check: e2e-shards (3)
- GitHub Check: e2e-shards (1)
- GitHub Check: e2e-shards (2)
- GitHub Check: e2e-shards (4)
🔇 Additional comments (2)
packages/models/src/models/meta.ts (2)
255-255: LGTM! Vision support enabled as intended.This change aligns with the PR objectives to enable vision support for Llama 4 Scout 17B.
272-273: Verify AWS Bedrock pricing and vision support for Llama 4 Maverick-17B instruct
- inputPrice updated to $0.24/1M tokens, outputPrice to $0.97/1M tokens
- vision support enabled
Confirm these match the official AWS Bedrock on-demand pricing formeta.llama4-maverick-17b-instruct-v1:0.
| { | ||
| providerId: "aws-bedrock", | ||
| modelName: "meta.llama3-1-8b-instruct-v1:0", | ||
| inputPrice: 2.2 / 1e6, | ||
| outputPrice: 6.6 / 1e6, | ||
| requestPrice: 0, | ||
| contextSize: 128000, | ||
| maxOutput: 2048, | ||
| streaming: true, | ||
| vision: false, | ||
| tools: false, | ||
| }, |
There was a problem hiding this comment.
Fix critical pricing error for aws-bedrock llama-3.1-8b-instruct.
The pricing is off by 100× and uses asymmetric rates when AWS Bedrock pricing is symmetric. According to past review verification, AWS charges $0.22 per 1,000 tokens for both input and output.
Additionally, maxOutput should be 128000, not 2048.
Apply this diff to correct the pricing and maxOutput:
{
providerId: "aws-bedrock",
modelName: "meta.llama3-1-8b-instruct-v1:0",
- inputPrice: 2.2 / 1e6,
- outputPrice: 6.6 / 1e6,
+ inputPrice: 0.22 / 1e3,
+ outputPrice: 0.22 / 1e3,
requestPrice: 0,
contextSize: 128000,
- maxOutput: 2048,
+ maxOutput: 128000,
streaming: true,
vision: false,
tools: false,
},Based on past review verification results.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| providerId: "aws-bedrock", | |
| modelName: "meta.llama3-1-8b-instruct-v1:0", | |
| inputPrice: 2.2 / 1e6, | |
| outputPrice: 6.6 / 1e6, | |
| requestPrice: 0, | |
| contextSize: 128000, | |
| maxOutput: 2048, | |
| streaming: true, | |
| vision: false, | |
| tools: false, | |
| }, | |
| { | |
| providerId: "aws-bedrock", | |
| modelName: "meta.llama3-1-8b-instruct-v1:0", | |
| inputPrice: 0.22 / 1e3, | |
| outputPrice: 0.22 / 1e3, | |
| requestPrice: 0, | |
| contextSize: 128000, | |
| maxOutput: 128000, | |
| streaming: true, | |
| vision: false, | |
| tools: false, | |
| }, |
🤖 Prompt for AI Agents
In packages/models/src/models/meta.ts around lines 11 to 22, the aws-bedrock
llama3 entry has pricing off by 100× and asymmetric input/output rates and an
incorrect maxOutput; update inputPrice and outputPrice to the symmetric AWS rate
of 0.22 per 1,000 tokens (e.g. 0.22 / 1000) and change maxOutput from 2048 to
128000, leaving requestPrice and the other fields unchanged.
1 participant
Summary
New Models Added
meta.llama3-8b-instruct-v1:0)meta.llama3-70b-instruct-v1:0)meta.llama3-1-8b-instruct-v1:0)meta.llama3-1-70b-instruct-v1:0)meta.llama3-2-1b-instruct-v1:0)meta.llama3-2-3b-instruct-v1:0)meta.llama3-2-11b-instruct-v1:0) - with visionmeta.llama3-2-90b-instruct-v1:0) - with visionmeta.llama3-3-70b-instruct-v1:0)Updated Models
Test plan
aws bedrock list-foundation-models --region us-east-1to verify modelspnpm format🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Chores