Bedrock model discovery and validation by stadgel-tc2 · Pull Request #9997 · aaif-goose/goose

stadgel-tc2 · 2026-06-25T07:20:40Z

Summary

This PR improves Amazon Bedrock model discovery and validation so Bedrock behaves more like other Goose providers.

Adds dynamic Bedrock model discovery via ListFoundationModels and ListInferenceProfiles.
Registers Bedrock with provider inventory refresh so model lists can update from AWS.
Filters discovered models to chat-capable options and excludes embedding/rerank models such as Cohere Embed.
Removes the static US-only model list as the primary source of selectable models.
Fixes Bedrock ConverseStream error handling so access/validation failures fail fast instead of retrying for a long time.
Adds model invocation validation when selecting models in CLI/server/UI flows.
Updates the desktop model picker to show a “Checking model…” state and keep the modal open on validation failure.

Details

Previously, the Bedrock provider used a hardcoded list of mostly us.* model IDs. This meant users in other regions could not reliably select valid regional inference profiles, and the picker could miss models available to their configured AWS account.

This change adds a Bedrock control-plane client and discovers:

active system/application inference profile IDs
on-demand foundation models with text input/output and streaming support
The discovered list is deduped, sorted, and filtered to remove non-chat models, including embed/rerank profiles that may otherwise appear in Bedrock discovery APIs.

The PR also validates selected models by invoking a small provider test before saving/applying the model in the server-backed UI paths. This catches AWS-side model access restrictions earlier, instead of failing only when the user sends their first chat message.

Testing

Ran Bedrock provider unit tests with aws-providers enabled.
Verified CLI goose configure lists dynamically discovered Bedrock models.
Verified embed/rerank models are excluded from the selectable list.
Verified inaccessible Bedrock models fail quickly during configuration/model selection.
Verified desktop UI model selection validates the model before closing the modal.

Related Issues

Not known.

Screenshots/Demos (for UX changes)

Before:

After:

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00272832ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0475cc0d30

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…he Converse path 2)Apply the probe timeout to stream creation too

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db7072c93f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-25T09:45:56Z

+                        let id = summary.inference_profile_id();
+                        if Self::is_chat_capable_model_id(id) {
+                            ids.insert(id.to_string());


Return ARNs for application inference profiles

When profile_type is Application, this stores the short inference_profile_id, but Bedrock requires application inference profiles to be invoked by ARN; only system-defined cross-region profiles can use the short ID. Any discovered application profile will therefore be shown/selectable as a model but later ConverseStream validation fails with an invalid model identifier. Use summary.inference_profile_arn() for application profiles, or return ARNs for all profiles.

Useful? React with 👍 / 👎.

Support for bedrock application inference profiles could be the subject of another PR.

clouatre · 2026-06-25T13:05:04Z

Good work here. A few notes from the authors of #9707 (Bedrock Mantle support), which this PR builds on:

The rename from BEDROCK_KNOWN_MODELS to BEDROCK_MANTLE_MODELS is the right separation of concerns: the static list was always Mantle-specific, and naming it as such makes the distinction between Mantle routing and standard Converse model IDs explicit.
The ResourceNotFoundException -> ExecutionError fix in both converse() and converse_stream() addresses a real hang (6 retries, ~2 min wait on invalid model names). We filed that independently as a standalone bug so it can be tracked separately from this PR's model-discovery scope.
Issue bug(bedrock): google.gemma-4-* models not reachable via aws_bedrock provider #10004 (google.gemma-4-* models not reachable via aws_bedrock) is a follow-on that will need to rebase onto this PR, since both touch bedrock.rs. We will coordinate the rebase once Bedrock model discovery and validation #9997 lands.

DOsinga · 2026-06-30T19:26:34Z

Thanks for this — the model-discovery direction is solid, and clouatre's endorsement from the Mantle side is reassuring.

Before we go deep on review, I want to flag a timing issue. @jamadeo is mid-flight on a fairly large refactor that moves providers out of the goose crate into the dedicated goose-providers crate, and is introducing a declarative provider model on top of it. The relevant in-progress PRs:

move anthropic provider into goose-providers #9985 / move OpenAI provider into goose-providers #9880 / Split provider inventory out of providers #9826 — moving providers and the inventory machinery into goose-providers
move ollama provider into goose-providers #9986 — moving the Ollama provider over (a good template for what a migrated provider looks like)
add declarative provider support to goose-providers crate #9992 — declarative provider support (DeclarativeProviderConfig, engine-based JSON configs)
add support for goose-providers to goose-sdk #10027 — a provider example built on top

This PR touches exactly the surfaces that refactor is reshaping: provider registration/inventory (init.rs, registrations.rs), fetch_supported_models, and the shared provider_test flow. If it lands first it'll collide head-on, and if Jack's work lands first this would need a non-trivial rebase into the new layout.

Could you take a look at Jack's branches above — especially #9986 and #9992 — and sketch how Bedrock discovery would slot into the new world? Bedrock is a bit special since it's the AWS SDK rather than an HTTP/REST shape, so it likely won't be a pure declarative JSON provider, but the fetch_supported_models + inventory-refresh wiring is precisely what's being reworked. Knowing how the two-client control-plane discovery fits the migrated goose-providers structure would help us land this cleanly rather than twice.

I'm going to snooze this for a few days and loop Jack in so you two can align on sequencing.

jamadeo · 2026-06-30T19:37:37Z

FWIW, I don't see an immediate conflict with the provider refactor. The relevant parts of inventory/models are merged, so as long as you're in sync with you should be good there. Soon enough we'll move the bedrock provider itself into goose-providers, but that hasn't started yet.

michaelneale · 2026-06-30T23:16:03Z

thanks @stadgel-tc2 - this looks fine to me. I know @lifeizhou-ap is workign flat out in some of these areas so some conflicts not expected. Are you able to resolve these (I can resolve but not confident in hand testing, so ideally would get the ok from you that it is valid/works). But once clean - I think can merge it in. I will pre-approve this so once it is green it gets in.

michaelneale

need to resolve conflicts and get a thumbs up from @stadgel-tc2 - but seems sensible to me, thorough improvement.

…n Bedrock model switching and provider model discovery.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6edd140d47

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

stadgel-tc2 · 2026-07-01T11:40:52Z

Thanks for the review @michaelneale . I reviewed and resolved the conflicts, from my side this is ok.

The summary of the resolve is :

Conflict resolutions:

crates/goose-server/src/routes/config_management.rs
Kept the provider/model validation work. CheckProviderRequest now accepts an optional model, and /config/check_provider validates the exact provider+model with test_provider_model() when a model is supplied. Without a model, it still falls back to checking the provider default.

crates/goose/tests/providers.rs
Resolved around provider test execution/session handling. The final result keeps the updated provider test path with session ID propagation and timeout behavior.

ui/desktop/src/components/ModelAndProviderContext.tsx
Kept the ACP model-switch path, but added validate-before-apply. changeModel() now calls validateProviderModel(providerName, modelName) first, then applies the session model through ACP or saves defaults. The error phase now distinguishes validation from agent/config application. Adopted upstream provider type import.

ui/desktop/src/components/settings/models/SwitchModelModal.tsx
Resolved the modal submit behavior with upstream’s current APIs and the branch UX improvements: submitting state, close-on-success behavior, and model switch error handling.

Cargo.lock:
kept both upstream TLS dependency changes and Bedrock SDK dependency.

Some other follow-on changes:

crates/goose/src/acp/server.rs
Removes live provider-model validation from ACP session/default model update handlers. Validation is now done before applying from the UI/server REST path.

crates/goose/src/acp/server/config.rs
Allows refresh-backed providers to accept models not already in the cached list, and removes live provider-model validation from ACP default save.

ui/desktop/src/components/settings/models/modelInterface.ts
Adds validateProviderModel() using REST checkProvider; switches model discovery/reasoning metadata back to REST endpoints instead of ACP model listing, while using upstream ACP local inference and shared provider types.

ui/desktop/src/acp/providers.ts
Improves ACP error normalization so UI toasts show useful server/ACP error messages instead of generic failures.

ui/desktop/openapi.json and ui/desktop/src/api/types.gen.ts:
Generated API updates for the new optional model field on CheckProviderRequest.

crates/goose/tests/acp_custom_requests_test.rs and crates/goose/tests/acp_fixtures/server.rs:
Fix a race in the steer-session ACP test by waiting for real session updates instead of polling.

ui/desktop/src/i18n/messages/*.json:
Generated/updated translations for switchModelModal.checkingModel.

lifeizhou-ap · 2026-07-01T23:33:46Z

Hi @stadgel-tc2,

We have to revert this PR from main branch as it has reference to the goosed (goose-server). more info in the revert pr description.

Once our release is launched, would you please swap to use acp methods. We are sorry for the inconvenience.

* main: Revert "Bedrock model discovery and validation" (#10176) Improve ACP auth and origin defaults (#9886) Bedrock model discovery and validation (#9997) chore(deps): bump actions-rust-lang/setup-rust-toolchain from 1.16.1 to 1.17.0 (#10052)

stadgel-tc2 added 2 commits June 25, 2026 08:52

Improve Bedrock model discovery and validation

61cd6f6

Fix model switch error display

0027283

chatgpt-codex-connector Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread crates/goose/src/providers/provider_test.rs

stadgel-tc2 added 2 commits June 25, 2026 10:30

Fix Bedrock CI and i18n validation failures

f29f58e

Update English desktop i18n messages

0475cc0

chatgpt-codex-connector Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread crates/goose/src/providers/bedrock.rs Outdated

Comment thread crates/goose/src/providers/provider_test.rs Outdated

Implemented Codex Bot Suggestions: 1)Keep GPT-OSS Bedrock models on t…

db7072c

…he Converse path 2)Apply the probe timeout to stream creation too

chatgpt-codex-connector Bot reviewed Jun 25, 2026

View reviewed changes

clouatre mentioned this pull request Jun 25, 2026

bug(bedrock): google.gemma-4-* models not reachable via aws_bedrock provider #10004

Open

clouatre mentioned this pull request Jun 25, 2026

bug(bedrock): ResourceNotFoundException retried 6 times, ~2 min hang on invalid model name #10006

Closed

DOsinga added the snoozed:20260703 label Jun 30, 2026

DOsinga assigned jamadeo Jun 30, 2026

DOsinga assigned michaelneale Jun 30, 2026

michaelneale approved these changes Jun 30, 2026

View reviewed changes

stadgel-tc2 added 2 commits July 1, 2026 09:47

Resolve merge conflicts and validate model switches

2c894ad

Merge upstream main into bedrock-model-discovery. Resolve conflicts i…

6edd140

…n Bedrock model switching and provider model discovery.

chatgpt-codex-connector Bot reviewed Jul 1, 2026

View reviewed changes

Comment thread crates/goose/src/providers/inventory/registrations.rs

Abhijay007 added this pull request to the merge queue Jul 1, 2026

Merged via the queue into aaif-goose:main with commit 7026e14 Jul 1, 2026
24 checks passed

lifeizhou-ap mentioned this pull request Jul 1, 2026

Revert "Bedrock model discovery and validation" #10176

Merged

github-actions Bot mentioned this pull request Jul 2, 2026

chore(release): release version 1.41.0 #10199

Closed

DOsinga mentioned this pull request Jul 5, 2026

Restore Bedrock model discovery using ACP #10201

Closed

Uh oh!

Conversation

stadgel-tc2 commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details

Testing

Related Issues

Screenshots/Demos (for UX changes)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

stadgel-tc2 Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

clouatre commented Jun 25, 2026

Uh oh!

DOsinga commented Jun 30, 2026

Uh oh!

jamadeo commented Jun 30, 2026

Uh oh!

michaelneale commented Jun 30, 2026

Uh oh!

michaelneale left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

stadgel-tc2 commented Jul 1, 2026

Uh oh!

Uh oh!

lifeizhou-ap commented Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

stadgel-tc2 commented Jun 25, 2026 •

edited

Loading