MCP Server OAuth Discovery doesn't work.

[gazzadownunder]

Follow up issue for:

MCP auth doesn't work with Keycloak. #4611 and merge #5458

Describe the bug

The Goose Client doesn't follow the correct OAuth Discovery process flow correctly.

PR #511 (merged Nov 5, 2025) DOES implement RFC 9728 (Protected Resource Metadata Discovery) in the rmcp library. The PR:

1. ✅ Adds discover_resource_metadata_url() method
2. ✅ Implements two-step discovery:
   • First: Queries /.well-known/oauth-protected-resource on MCP server
   • Then: Extracts authorization_servers array
   • Finally: Queries authorization server metadata
3. ✅ Includes fallback discovery paths
4. ✅ Targets version 0.8.5

Current flow

The rmcp library's OAuthState::new():

• ✅ Receives the 401 response (handled by transport layer)
• ❌ SKIPS Step 2 (protected resource metadata)
• ❌ Directly queries {mcp_server_url}/.well-known/oauth-authorization-server (Step 3, but at wrong URL)
• ❌ Assumes MCP server IS the authorization server

The Problem

Version 0.8.5 is NOT yet published to crates.io!

• Latest published version: 0.8.3 (per docs.rs)
• Goose is using: 0.8.3 (Cargo.toml:18)
• PR Adds 'instructions' field to InitializeResult #511 targets: 0.8.5 (not yet released)

Conclusion

1. ❌ The fix is NOT yet published to crates.io as version 0.8.5

2. ❌ Goose is using the older 0.8.3 which lacks RFC 9728 support

3. ❌ OAuth code might require updating to point to the authorisation server, rather than the MCP server

4. ✅ Once rmcp 0.8.5 is published, Goose can update
   Current Operation:

5. Client tries to connect → gets 401 ✅

6. rmcp 0.8.3 doesn't use protected-resource metadata ✅

7. Tries /.well-known/oauth-authorization-server on MCP server ✅

8. Fails because it doesn't have authorization_servers ✅

MCP Specification Compliance Analysis

Overview

This appendix compares Goose's OAuth implementation with the MCP OAuth 2.0 Authorization Specification to identify compliance gaps and differences.

Summary of Findings (Updated 2025-11-11)

Status: ❌ Non-Compliant - OAuth discovery does not follow MCP specification

Goose's OAuth implementation has two separate flows:

1. MCP servers (mod.rs) - Uses rmcp library for OAuth
2. Databricks (providers/oauth.rs) - Custom Databricks-specific implementation

Critical Issue: The MCP OAuth flow (via rmcp library v0.8.3) is NOT compliant with the MCP OAuth specification. It implements RFC 8414 (Authorization Server Metadata) but does NOT implement RFC 9728 (Protected Resource Metadata), which is required by the MCP specification.

Detailed Comparison

1. OAuth Discovery Endpoints (MCP Servers)

Aspect	MCP Specification	rmcp Library (v0.8.3)	Goose/rmcp Implementation	Compliant?
Protected Resource Discovery	/.well-known/oauth-protected-resource (RFC 9728)	❌ Not implemented	❌ Skipped (mod.rs:76)	NO
Authorization Server Discovery	/.well-known/oauth-authorization-server (RFC 8414)	✅ Implemented	✅ Queries {mcp_server_url}/.well-known/oauth-authorization-server	⚠️ PARTIAL
Discovery sequence	Two-step: resource → auth server	Single-step: auth server only	Single-step: queries MCP server directly	NO
WWW-Authenticate header parsing	Extract protected resource metadata URL	✅ Detects 401	❌ Ignores metadata URL	NO

Analysis:

The MCP specification requires a two-step discovery process:

1. Step 1: Client connects to MCP server → receives 401 Unauthorized with WWW-Authenticate header

   HTTP/1.1 401 Unauthorized
   WWW-Authenticate: Bearer realm="MCP Server",
     oauth_protected_resource_metadata="https://mcp.example.com/.well-known/oauth-protected-resource"

2. Step 2: Client requests protected resource metadata from MCP server

   GET https://mcp.example.com/.well-known/oauth-protected-resource
   
   Response:
   {
     "resource": "https://mcp.example.com",
     "authorization_servers": ["https://auth.example.com"]
   }

3. Step 3: Client requests authorization server metadata from the authorization server (NOT the MCP server)

   GET https://auth.example.com/.well-known/oauth-authorization-server
   
   Response:
   {
     "authorization_endpoint": "https://auth.example.com/authorize",
     "token_endpoint": "https://auth.example.com/token",
     ...
   }

What rmcp/Goose Actually Does (mod.rs:76):

let mut oauth_state = OAuthState::new(mcp_server_url, None).await?;

@zhaohuabing @romixch @botengyao @gazzadownunder @alexhancock

[alexhancock]

Thanks for filing and identifying we have not yet upgraded to 0.8.5 of the rust-sdk. I thought we had. I will do that now.

Beyond that, let's discuss rust-sdk's support for oauth RFC 9728 and protected resource discovery.

My understanding is this (and associated) code implements support: https://github.com/modelcontextprotocol/rust-sdk/blob/03040f8b0bfb9ac43e73234189d409da67e1fe5b/crates/rmcp/src/transport/auth.rs#L678

That's not to say it's without issue... is there an issue with how we're handling the case where a WWW-Authenticate header is used?

I'll upgrade the dependency to get some of this working now, and let's discuss if there is more to do.

[alexhancock]

goose now depends on rmcp 0.8.5 which has an improved posture for handling some of the mentioned things

it will roll out in 1.14.0 of goose

#5676

[gazzadownunder]

Thanks for filing and identifying we have not yet upgraded to 0.8.5 of the rust-sdk. I thought we had. I will do that now.

Beyond that, let's discuss rust-sdk's support for oauth RFC 9728 and protected resource discovery.

My understanding is this (and associated) code implements support: https://github.com/modelcontextprotocol/rust-sdk/blob/03040f8b0bfb9ac43e73234189d409da67e1fe5b/crates/rmcp/src/transport/auth.rs#L678

That's not to say it's without issue... is there an issue with how we're handling the case where a WWW-Authenticate header is used?

I'll upgrade the dependency to get some of this working now, and let's discuss if there is more to do.

An additional change will be required for (mod.rs:76):

let mut oauth_state = OAuthState::new(mcp_server_url, None).await?;

It currently references the MCP server URL, it should be the IDP Authorisation server.

[alexhancock]

@gazzadownunder I'm not sure I understand. The lines following that call start_authorization which goes through the discovery process in rmcp to get the authorization server url, and then we use it.

    let mut oauth_state = OAuthState::new(mcp_server_url, None).await?;
    let redirect_uri = format!("http://localhost:{}/oauth_callback", used_addr.port());
    oauth_state
        .start_authorization(&[], redirect_uri.as_str(), Some("goose"))
        .await?;

    let authorization_url = oauth_state.get_authorization_url().await?;
    if webbrowser::open(authorization_url.as_str()).is_err() {
        eprintln!("Open the following URL to authorize {}:", name);
        eprintln!("  {}", authorization_url);
    }

I'm maintaining a lot of this MCP related code, but by no means an oauth expert! So let me know if there is a flaw I am not understanding. I'd also be totally open to a PR to either goose or rmcp if you want to show what is needed.

[gazzadownunder]

Hi @alexhancock,

Here are the results of my testing based on rmcp v0.8.5.

IDP Discovery Issue:

rmcp v0.8.5 has improved things, but there are still issues with the discovery process not following the standard. However, I think this is an issue with the RMCP SDK as the MCP Inspector is showing the same behaviour. When the server returns the 401 error, with the www-authenticate header including the resource_metadata, however, the client doesn't try to retrieve the protected resource metadata from the URL, instead it defaults to the fallback discovery behaviour of trying to append different combinations of URLs until it gets a response.

https://modelcontextprotocol.io/specification/draft/basic/authorization

Having just read this again, it does say the client must implement one of these discovery methods. However, I'm not sure why it defaults to guessing the url for the authorisation server and when the server tells you the correct URL.

Default Client_ID

The rmcp library has a default value of "mcp-client" assigned to the client_id. Once goose has found the authorisation server, it tries to authenticate with the client_id=mcp-client, which fails, unless you happen to be using the same client_id. I've also tried with a DCR enabled MCP server, but it looks like goose doesn't support DCR at the moment.

As the client is not using DCR and there are still a large number of OAuth MCP servers that don't support DCR, the simplest solution to address this one is to include a client_id and scope fields on the extension dialog to allow the user to manually define the details. This is the same approach as used by MCP Inspector.

[alexhancock]

Apologies if I am missing something, but the discussion seems to be going in many different directions....

I don't see the current state of the code the same way as posted above. https://github.com/modelcontextprotocol/rust-sdk/blob/main/crates/rmcp/src/transport/auth.rs implements proper support for Protected Resource Metadata discovery (RFC9728) and this implementation is used in goose.

Client ID and scope configuration is a good idea, but sort of separate from what the original requests were in this issue, which I believe to now be resolved.

If there is a specific PR you'd like to raise to show something you'd like changed, or a server that doesn't work with goose's auth code we should look at, please open a new issue and we'll be happy to get into it.

[gazzadownunder]

Hi @alexhancock

I've been looking at the issues, and here are some details on how to resolve them.

I started looking at the MCP Inspector, as it has the same issue with the discovery of not using the protected_resource. On the Typescript SDK, the solution is to add authProvider to the TransportOptions and the protected_resource details are using for discovery. The Rust SDK has the same issue however, the fix didn't work. Here is Claude's analysis of the issue:

OAuth Resource Metadata Discovery Issue in Goose MCP Client

Executive Summary

The Goose Rust MCP client has the same OAuth discovery issue as the TypeScript MCP Inspector - it does not properly parse and use the resource_metadata field from WWW-Authenticate headers for automatic OAuth endpoint discovery. However, the fix requires understanding how the Rust rmcp SDK handles resource metadata, as it uses a different API than the TypeScript SDK.

Background

The Problem (Same as MCP Inspector)

When an MCP server returns a 401 Unauthorized response with a WWW-Authenticate header containing resource_metadata, the client should:

1. Extract the www_authenticate_header from the 401 response
2. Parse the header to find the resource_metadata parameter
3. Base64-decode the resource_metadata JSON
4. Use the decoded metadata for OAuth endpoint discovery (authorization URL, token URL, etc.)

Current Behavior in Goose

Location: crates/goose/src/agents/extension_manager.rs:443-466

// Initial connection WITHOUT auth
let transport = StreamableHttpClientTransport::with_client(
    client,
    StreamableHttpClientTransportConfig {
        uri: uri.clone().into(),
        ..Default::default()
    },
);
let client_res = McpClient::connect(transport, timeout, provider).await;

// Detect auth error
let client = if let Some(_auth_error) = extract_auth_error(&client_res) {
    // ❌ PROBLEM: _auth_error is discarded!
    // The www_authenticate_header containing resource_metadata is lost

    let am = oauth_flow(uri, name).await?;

    // Reconnect with auth
    let client = AuthClient::new(reqwest::Client::default(), am);
    let transport = StreamableHttpClientTransport::with_client(
        client,
        StreamableHttpClientTransportConfig {
            uri: uri.clone().into(),
            ..Default::default()
        },
    );
    McpClient::connect(transport, timeout, provider).await?
} else {
    client_res?
};

Location: crates/goose/src/oauth/mod.rs:76

pub async fn oauth_flow(
    mcp_server_url: &String,
    name: &String,
) -> Result<AuthorizationManager, anyhow::Error> {
    // ... credential caching logic ...

    // ❌ PROBLEM: Passing None instead of resource_metadata
    let mut oauth_state = OAuthState::new(mcp_server_url, None).await?;

    oauth_state
        .start_authorization(&[], redirect_uri.as_str(), Some("goose"))
        .await?;
    // ...
}

The TypeScript Fix (For Comparison)

In the TypeScript MCP Inspector, the fix was simple:

Before:

const transport = new StreamableHTTPClientTransport(serverUrl, {
    // ❌ No authProvider passed
});

After:

const transport = new StreamableHTTPClientTransport(serverUrl, {
    authProvider: serverAuthProvider  // ✅ SDK auto-handles everything
});

The TypeScript SDK automatically:

• Intercepts 401 responses
• Parses WWW-Authenticate header
• Extracts and decodes resource_metadata
• Uses it for OAuth discovery

Attempted Rust Fix (Incomplete)

What Was Attempted

1. Added parsing function (crates/goose/src/oauth/mod.rs:21-42)

use base64::Engine;

/// Parse the WWW-Authenticate header to extract resource_metadata
/// The header format is: Bearer resource_metadata="<base64_encoded_json>"
pub fn parse_www_authenticate_header(www_authenticate_header: &str) -> Option<String> {
    let resource_metadata_prefix = "resource_metadata=\"";

    if let Some(start_idx) = www_authenticate_header.find(resource_metadata_prefix) {
        let start = start_idx + resource_metadata_prefix.len();
        if let Some(end_idx) = www_authenticate_header[start..].find('"') {
            let encoded = &www_authenticate_header[start..start + end_idx];

            // Decode the base64-encoded JSON
            if let Ok(decoded_bytes) = base64::engine::general_purpose::STANDARD.decode(encoded) {
                if let Ok(decoded_str) = String::from_utf8(decoded_bytes) {
                    return Some(decoded_str);
                }
            }
        }
    }

    None
}

✅ Status: This parsing function is correct and ready to use.

2. Attempted to extract and use resource_metadata (crates/goose/src/agents/extension_manager.rs:443-449)

let client = if let Some(auth_error) = extract_auth_error(&client_res) {
    // Extract the WWW-Authenticate header
    let resource_metadata = crate::oauth::parse_www_authenticate_header(
        &auth_error.www_authenticate_header,
    );

    // ❌ PROBLEM: How to pass resource_metadata to oauth_flow()?
    let am = oauth_flow(uri, name, resource_metadata).await?;
    // ...
}

⚠️ Status: Code written but blocked by API incompatibility.

Why It Didn't Work

The API Mismatch:

// Tried to add resource_metadata parameter:
pub async fn oauth_flow(
    mcp_server_url: &String,
    name: &String,
    resource_metadata: Option<String>,  // ❌ Added this
) -> Result<AuthorizationManager, anyhow::Error> {
    // Tried to pass it to OAuthState::new():
    let mut oauth_state = OAuthState::new(mcp_server_url, resource_metadata).await?;
    // ...
}

Compilation Error:

error[E0308]: mismatched types
   --> crates\goose\src\oauth\mod.rs:101:59
    |
101 |     let mut oauth_state = OAuthState::new(mcp_server_url, resource_metadata).await?;
    |                           ---------------                 ^^^^^^^^^^^^^^^^^
    |                           expected `Option<Client>`, found `Option<String>`
    |
note: associated function defined here
   --> rmcp-0.8.5\src\transport\auth.rs:969:18

The Real Signature:

// From rmcp SDK (version 0.8.5)
impl OAuthState {
    pub async fn new<U: IntoUrl>(
        url: U,
        client: Option<reqwest::Client>,  // ← Not resource_metadata!
    ) -> Result<Self, Error>;
}

The second parameter is for an optional HTTP client, NOT resource_metadata.

Key Differences: TypeScript SDK vs Rust SDK

Aspect	TypeScript SDK	Rust rmcp SDK
Auth provider passing	authProvider in transport config	AuthClient wrapper around HTTP client
Automatic 401 handling	✅ Yes, when authProvider present	❌ No, manual detection required
WWW-Authenticate parsing	✅ Automatic	❌ Not automatic
Resource metadata usage	✅ Automatic discovery	❓ Unknown API
Architecture	Transport handles auth	Application handles auth

The Unanswered Question

How does the Rust rmcp SDK (v0.8.5) use resource_metadata for OAuth discovery?

Possible answers:

1. Different method on OAuthState - e.g., with_metadata(), set_metadata(), or passed to start_authorization()
2. Not supported yet - The feature may not be implemented in rmcp 0.8.5
3. Different architecture - May require parsing and manually constructing OAuth URLs

Files Modified (Partial Implementation)

1. crates/goose/src/oauth/mod.rs

• ✅ Added base64::Engine import (line 5)
• ✅ Added parse_www_authenticate_header() function (lines 21-42)
• ⚠️ oauth_flow() signature currently has only 2 parameters (needs fixing)

2. crates/goose/src/agents/extension_manager.rs

• ⚠️ Changed _auth_error to auth_error (line 443)
• ⚠️ Added resource_metadata extraction (lines 444-447)
• ❌ Tries to call oauth_flow(uri, name, resource_metadata) - WILL NOT COMPILE

3. .cargo/config.toml (Build fixes)

[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "link-arg=advapi32.lib"]

[env]
CMAKE = "C:\\Program Files\\CMake\\bin\\cmake.exe"

Build Status

Current compilation error:

error[E0308]: mismatched types
   --> crates\goose\src\agents\extension_manager.rs:449
    |
449 |     let am = oauth_flow(uri, name, resource_metadata).await?;
    |              ----------                 ^^^^^^^^^^^^^^^^^ expected 2 arguments, found 3

The code will not compile until the correct rmcp API is identified.

rmcp SDK Information

Version: 0.8.5
Dependencies: Listed in Cargo.toml

rmcp = { version = "0.8.5", features = ["schemars", "auth"] }

Relevant Types:

• rmcp::transport::auth::OAuthState - OAuth state machine
• rmcp::transport::auth::AuthClient - HTTP client wrapper with auth
• rmcp::transport::AuthorizationManager - Token manager
• rmcp::transport::streamable_http_client::AuthRequiredError - Contains www_authenticate_header: String

Next Steps Required

Option 1: Find the Correct rmcp API

1. Research rmcp SDK v0.8.5 source code or documentation
2. Look for how resource_metadata is intended to be used:
   • Check OAuthState methods besides new()
   • Check if start_authorization() accepts metadata
   • Check for builder patterns or configuration structs
3. Implement the correct approach
4. Complete the build and test

Option 2: Verify Feature Support

1. Check if rmcp v0.8.5 supports resource_metadata at all
2. If not supported:
   • Consider upgrading rmcp version
   • Or contribute the feature to rmcp SDK
   • Or document as "not yet supported"

Option 3: Manual Implementation

1. Parse resource_metadata to extract OAuth endpoints
2. Manually construct OAuthState using discovered endpoints
3. May require understanding OAuthState internals

References

Code Locations

• Extension Manager: crates/goose/src/agents/extension_manager.rs:428-466
• OAuth Flow: crates/goose/src/oauth/mod.rs:55-103
• Auth Error Extraction: crates/goose/src/agents/extension_manager.rs:224-244
• AuthRequiredError struct: Contains field www_authenticate_header: String

Related Files

• crates/goose/src/oauth/persist.rs - Credential caching
• crates/goose/examples/databricks_oauth.rs - OAuth example (may have clues)

MCP OAuth Specification

The resource_metadata field in the WWW-Authenticate header is part of the MCP OAuth specification. It contains a base64-encoded JSON object with:

{
  "authorization_endpoint": "https://...",
  "token_endpoint": "https://...",
  "scopes_supported": [...],
  // ... other OAuth discovery metadata
}

This allows servers to provide OAuth configuration dynamically without requiring pre-configuration on the client side.

Conclusion

The issue is confirmed and the parsing logic is ready, but the implementation is blocked by uncertainty about the rmcp SDK API. The Rust SDK requires a different approach than the TypeScript SDK's simple authProvider parameter.

Critical Question: How does rmcp::transport::auth::OAuthState accept and use resource_metadata for OAuth endpoint discovery?

[gazzadownunder]

Hi @alexhancock,

The second issue of the client_id is not being set, and it is using the default value. I managed to update the extension dialog to support the extra client_id and scope fields and updated extension_manager.rs to pass these values to the RMCP SDK. However, there are other issues in the SDK, and these values are not used. It requires a change to the SDK to support the passing of a user-defined client_id.

                .await;
                let client = if let Some(_auth_error) = extract_auth_error(&client_res) {
                    let am = oauth_flow(uri, name, client_id.as_ref(), scopes)
                        .await
                        .map_err(|e| {
                            ExtensionError::SetupError(format!("OAuth authentication failed: {}", e))
                        })?;
                    let client = AuthClient::new(reqwest::Client::default(), am);
                    let transport = StreamableHttpClientTransport::with_client(
                        client,