feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange

[yassin-berriai]

TLDR

Problem this solves:

• LiteLLM cannot call Microsoft 365 Copilot (Graph Copilot Chat API)
• Copilot only accepts a delegated user token, so a gateway key is not enough

How it solves it:

• New microsoft_365_copilot chat provider
• New IdP-agnostic OAuth token exchange module, litellm/litellm_core_utils/oauth_token_exchange.py, with two grant profiles: rfc8693 (standard token exchange) and jwt_bearer_obo (JWT bearer on-behalf-of). The provider exchanges the caller's JWT for a delegated Graph token through it
• Add Model / credential form gets an Auth Type selector that shows only the fields for the chosen mode and states the IdP limitation: Graph only accepts Microsoft Entra tokens, so for this provider the token endpoint must be Entra (callers can still sign in through any IdP federated with Entra)
• The OAuth token exchange settings are server-owned (proxy-admin only, rejected in request bodies), so in Add Model that auth type is credentialOnly: the form shows "Create credential", which opens the credential form with the provider locked and the auth type preselected, saves an LLM credential, and attaches it via litellm_credential_name. Submitting the model with no credential attached is blocked
• The provider accepts max_tokens / max_completion_tokens for OpenAI-client compatibility (the UI "Test connection" health check injects max_tokens) and never sends them to Graph. microsoft_365_copilot/chat is in the cost map at $0 (Copilot is license-billed) so the Add Model model dropdown lists it
• /health/test_connection without a model id borrows the first deployment with the same model. A request that brings its own api_key now gets none of that deployment's stored auth settings (token_exchange_*, client_id/client_secret, litellm_credential_name, Azure AD, AWS role and so on). Before, a static-token test inherited a saved OAuth deployment's exchange config and failed asking for a caller JWT. This applies to every provider; re-testing a saved model, which sends its id and no key, is unchanged
• Graph's beta chat API returns some replies doubled back to back ("pongpong" for "reply with pong"), so a reply made of the exact same text twice is collapsed to one copy before it is returned and counted
• Claude Desktop sends a system turn after the user's question. That turn used to become the Graph prompt, so Copilot answered the environment block with a greeting. The prompt is now the last user message
• /health/test_connection now passes the caller's request headers to the probe as server-built secret_fields, the same way normal requests do, so a caller signed in with an IdP JWT can test an OAuth token exchange model. A caller without a JWT still gets the 401

User Flow

Before: an admin whose users sign in to LiteLLM with their IdP JWT cannot add Microsoft 365 Copilot as a model

1. Admin opens https://litellm-domain/ui/models-and-endpoints/ > Add Model and types "Microsoft 365" in Provider; the dropdown says "No providers found"
2. Admin instead adds model: microsoft_365_copilot/chat in config.yaml; the proxy logs LLM Provider NOT provided while loading it
3. A user sends POST https://litellm-domain/v1/chat/completions with their JWT and "model": "m365-copilot"; it returns 400 There are no healthy deployments for this model

After: the same admin adds the model from the UI, and the user's own Copilot license answers the request

1. Admin opens the same Add Model page, picks "Microsoft 365 Copilot", and picks chat from the model dropdown. Auth Type defaults to "OAuth token exchange (on-behalf-of)" with the note that the endpoint must be Entra for this provider. Clicking "Create credential" opens the credential form with that auth type preselected, showing Token Endpoint URL, Exchange Grant (jwt_bearer_obo default), Client ID, Client Secret, Scope (https://graph.microsoft.com/.default default) and Audience. Saving it attaches the credential to the model, and "Test connection" passes. Switching Auth Type to "Static delegated access token" shows only the token field inline
2. The user sends the same POST https://litellm-domain/v1/chat/completions with their Entra-issued JWT; it returns 200 with Copilot's answer in choices[0].message.content, also with "stream": true
3. A caller that sends a LiteLLM key instead of a JWT gets 401 microsoft_365_copilot with OAuth token exchange requires the caller's IdP-issued access token in the Authorization header

How it works

sequenceDiagram
    participant C as Caller (IdP JWT)
    participant L as LiteLLM proxy
    participant T as Configured token endpoint (Entra for this provider)
    participant G as Graph beta Copilot
    C->>L: POST /v1/chat/completions (Bearer JWT)
    L->>L: enable_jwt_auth validates the JWT
    L->>T: token_exchange_profile form (jwt_bearer_obo or rfc8693), client_secret_post
    T-->>L: delegated Graph token (cached until expires_in minus 60s)
    L->>G: POST /copilot/conversations, then /conversations/{id}/chat (delegated token)
    G-->>L: messages[] (user turn, then Copilot reply)
    L-->>C: OpenAI chat.completion (last message), or fake-streamed chunks

Loading

Exchange forms built by build_token_exchange_form(config, subject_token):

rfc8693:        grant_type=urn:ietf:params:oauth:grant-type:token-exchange
                subject_token=<jwt>  subject_token_type=urn:ietf:params:oauth:token-type:access_token
                scope=<if set>  audience=<if set>
jwt_bearer_obo: grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer
                assertion=<jwt>  scope=<required>  requested_token_use=on_behalf_of
both:           client_id, client_secret (client_secret_post)

Deployment params: token_exchange_endpoint, token_exchange_profile (default jwt_bearer_obo), token_exchange_scope (default https://graph.microsoft.com/.default), token_exchange_audience, client_id, client_secret. With endpoint, client ID and secret all set the provider exchanges the caller's JWT and fails closed (401) without one. With none of them set, api_key is used as a pre-acquired delegated Graph token. A partial set, an unknown profile, a non-HTTPS endpoint, or an empty jwt_bearer_obo scope is rejected (400) instead of falling back. The four token_exchange_* params are added to the proxy's banned request-body params, so a caller cannot point the server's client secret at its own endpoint. The Graph host is fixed and api_base is not honoured.

Only a proxy admin can set the four token_exchange_* params: they are part of server_owned_wif_litellm_params (oauth_token_exchange_litellm_params), so the existing non-admin gates on model writes (including config inherited through litellm_credential_name), credential writes, request bodies, health display, and client api_base overrides apply. A team admin therefore cannot point a deployment at an endpoint that would receive callers' JWTs. client_id and client_secret are also in _FEDERATION_SURFACE_FIELDS, so a team admin patching only those on a deployment that carries exchange config gets 403 too, while the same patch on an ordinary deployment is still allowed.

With token exchange configured, a 401 or 403 belongs to the caller (missing or rejected JWT, or a Graph rejection of that user's token), so Router.deployment_callback_on_failure and the async fallback handler _trigger_cooldown_for_failed_deployment return early through is_caller_scoped_auth_failure without counting a failure or cooling the shared deployment down. This covers exchange config set inline or through litellm_credential_name; 429, 5xx and deployments using a static api_key keep the normal cooldown

Replies depend on the caller's delegated identity, so microsoft_365_copilot is in RESPONSE_CACHE_EXCLUDED_PROVIDERS and caching_handler never reads or writes the response cache for it. Graph returns no token counts, so usage is estimated with token_counter over the request messages and the reply, which keeps TPM limits effective.

The token cache key is a SHA-256 over every config field plus the subject token. Token endpoint 429 and 5xx keep their status, other 4xx map to 401, and Graph statuses (for example the 403 for a user without a Copilot license) pass through. The JWT and client secret are redacted from errors.

_collapse_doubled_reply keeps the first half of a reply only when its two halves are identical; a reply that legitimately repeats itself exactly is shortened too, which was accepted as the tradeoff for working around the Graph bug

The last user message is the Graph prompt and every other message, in order, goes to Graph as additionalContext. microsoft_365_copilot/chat sets supports_mid_conversation_system, so the Anthropic adapter keeps a system turn sent after the prompt (Claude Desktop sends its environment block that way) as system context instead of turning it into the prompt. At least one user message with text content is required, and images are rejected. locationHint.timeZone defaults to UTC and can be set with time_zone.

Files changed

File	Change
litellm/litellm_core_utils/oauth_token_exchange.py	OAuthTokenExchangeConfig, build_token_exchange_form, sync and async exchange_token, token cache, status mapping, redaction
litellm/llms/microsoft_365_copilot/chat/handler.py	Mode selection (exchange vs delegated token), Graph conversation create and chat, sync, async, and fake streaming
litellm/llms/microsoft_365_copilot/chat/transformation.py	Microsoft365CopilotChatConfig, OpenAI messages to Graph chat body, Graph response to ModelResponse, Graph error parsing
litellm/llms/microsoft_365_copilot/common_utils.py	Caller JWT extraction from secret_fields, provider error type
litellm/llms/microsoft_365_copilot/__init__.py, chat/__init__.py	Package exports
litellm/main.py	Dispatch custom_llm_provider == "microsoft_365_copilot"; pass token_exchange_* kwargs through
litellm/litellm_core_utils/get_litellm_params.py, litellm/types/litellm_params.py, litellm/types/router.py	token_exchange_* params
litellm/types/utils.py (oauth_token_exchange_litellm_params)	token_exchange_* join server_owned_wif_litellm_params, so only proxy admins can set them
litellm/caching/caching_handler.py, litellm/constants.py	_is_response_cache_excluded gates response-cache reads and writes for RESPONSE_CACHE_EXCLUDED_PROVIDERS
litellm/proxy/... gate messages, litellm/router_utils/clientside_credential_handler.py	Gate wording covers OAuth token exchange; the api_base override clears the exchange config
litellm/utils.py, litellm/types/utils.py	ProviderConfigManager entry, LlmProviders.MICROSOFT_365_COPILOT
litellm/constants.py	Graph beta base URL, provider defaults, exchange cache safety margin, default time zone
litellm/proxy/public_endpoints/provider_create_fields.json	Add Model fields for both auth types
provider_endpoints_support.json	Chat completions supported; messages and responses not
ui/litellm-dashboard/src/components/add_model/provider_auth_types.ts	Per-provider auth type definitions (fields, required fields, description)
ui/litellm-dashboard/src/components/add_model/provider_specific_fields.tsx	Auth Type selector; unselected fields are unmounted and not submitted
ui/litellm-dashboard/src/components/model_add/CredentialModal.tsx	Editing a credential across auth types deletes the other type's stored values
ui/litellm-dashboard/src/components/provider_info_helpers.tsx	Provider enum, display name, logo, placeholder
litellm/proxy/common_utils/credential_hydration.py	client_id and client_secret join _FEDERATION_SURFACE_FIELDS, so writes to them go through the proxy-admin gate
litellm/router_utils/cooldown_handlers.py, litellm/router.py, litellm/router_utils/fallback_event_handlers.py	is_caller_scoped_auth_failure: OAuth exchange 401/403 skip failure counting and cooldown
litellm/proxy/health_endpoints/_health_endpoints.py	_request_inherits_config_credentials: a truthy request api_key means the connection test uses only its own credentials. The probe gets server-built secret_fields from the caller's headers, overriding any body value

Linear ticket

Resolves LIT-9306

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally: uv run pytest tests/unit/llms/microsoft_365_copilot tests/unit/litellm_core_utils/test_oauth_token_exchange.py tests/unit/proxy/auth/test_auth_utils.py tests/unit/proxy/management_endpoints/test_model_management_endpoints.py -v
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Tested on litellm 1.106.0 with Postgres, enable_jwt_auth: true (Entra JWKS, user_id_jwt_field: oid), and a real Entra tenant. The middle-tier app registration exposes access_as_user and holds delegated Graph permissions for Copilot Chat. The caller JWT belongs to a user with a Microsoft 365 Copilot license. Before used main a9b9700790 on port 4000; after used this branch on port 4001. Tokens are redacted.

model_list:
  - model_name: m365-copilot
    litellm_params:
      model: microsoft_365_copilot/chat
      token_exchange_endpoint: os.environ/M365_TOKEN_ENDPOINT   # https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token
      token_exchange_profile: jwt_bearer_obo
      token_exchange_scope: https://graph.microsoft.com/.default
      client_id: os.environ/M365_OBO_CLIENT_ID
      client_secret: os.environ/M365_OBO_CLIENT_SECRET
  - model_name: gpt-control
    litellm_params:
      model: openai/gpt-5.4-mini

Body for every leg: {"model":"<model>","messages":[{"role":"user","content":"<prompt>"}]}

Before (a9b9700)

1. Control model, curl -sS localhost:4000/v1/chat/completions -H "Authorization: Bearer $USER_JWT" -d @body_control.json: HTTP 200, "content":"pong"
2. Copilot model, same curl with m365-copilot: HTTP 400 There are no healthy deployments for this model; proxy log LLM Provider NOT provided. ... You passed model=microsoft_365_copilot/chat
3. Admin UI, Add Model > Provider > "Microsoft 365": "No providers found"

After, generic token exchange config above

1. Copilot model with the user JWT: HTTP 200, "content":"LiteLLM Copilot live QA passed.LiteLLM Copilot live QA passed."
2. Same with "stream": true: HTTP 200, same content, then finish_reason: "stop" and data: [DONE]
3. Admin UI: the Auth Type selector and IdP note render as in the screenshot above

After, earlier revision of this PR with the same OBO flow (Entra-named params)

1. Copilot model: HTTP 200 "content":"pongpong"; stream HTTP 200 with [DONE]
2. LiteLLM key instead of the JWT: HTTP 401
3. grep -c over the proxy log for the caller JWT and the client secret: 0 and 0
4. UI-created model, then Playground: the Copilot answer renders (screenshot above)
5. User with no Copilot license: Graph's 403 you don't have a valid license comes back as 403

The doubled text comes from Graph, not LiteLLM. A direct POST https://graph.microsoft.com/beta/copilot/conversations/{id}/chat with the same delegated token returns the reply twice in its last message, and the provider passes it through unchanged.

Trailing system turn from Claude Desktop (d3b99bf)

Request: POST /v1/messages to a claude-m365-copilot alias with messages: [{"role":"user","content":"whats 1+1"}, {"role":"system","content":"# Environment\n..."}], a system list and a tool, as Claude Desktop sends it

1. Before, on the shared dev gateway: "Got it. What can I help you with today?Got it. What can I help you with today?"
2. After, local proxy on this branch, non-streaming: {"content":[{"type":"text","text":"1 + 1 = **2**."}],"stop_reason":"end_turn","usage":{"input_tokens":52,"output_tokens":9}}
3. After, streaming: one text_delta with 1 + 1 = **2**, then message_stop

Test connection with its own api_key (a5dfffe)

A saved microsoft_365_copilot/chat deployment with token_exchange_endpoint, client_id and client_secret, then the Add Model static-token test:

curl -sS localhost:4317/health/test_connection -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"litellm_params":{"model":"microsoft_365_copilot/chat","custom_llm_provider":"microsoft_365_copilot","api_key":"dummy-static-token"},"model_info":{}}'

1. Before (7d7b2ee): "status":"error", microsoft_365_copilot with OAuth token exchange requires the caller's IdP-issued access token, and the result echoes the other deployment's token_exchange_endpoint
2. After: "status":"error" from Graph itself, Protocol 'Bearer' failed to validate because The token could not be read., so the probe used the request's own token and nothing from the saved deployment

Tests

• tests/unit/litellm_core_utils/test_oauth_token_exchange.py: exact rfc8693 and jwt_bearer_obo forms, audience only on rfc8693, empty OBO scope and non-HTTPS endpoint rejected, cache key separation for every config field and the subject token, TTL margin, 400 to 401, 429 and 5xx preserved, secret and subject token redaction, sync and async
• tests/unit/llms/microsoft_365_copilot: mode selection, partial config rejected without any HTTP call, Graph gets the exchanged token never the caller JWT, message mapping, stream chunks, error mapping, and a litellm.completion test that mocks only HTTP and asserts the exchange POST reaches the configured endpoint
• tests/unit/proxy/auth/test_auth_utils.py, model management, credential endpoint tests: token_exchange_endpoint in a request body is rejected; a team admin setting it directly or by attaching a credential that holds it gets 403; a proxy admin is allowed; a team admin rpm-only edit still works
• caching tests: identical sync, async and stream Copilot calls each reach Graph and nothing is cached; a control provider still hits the cache
• usage: prompt and completion tokens equal token_counter over the messages and reply, non-stream and stream
• an api_base override on an exchange deployment clears the exchange config and fails closed with no token-endpoint POST
• tests/unit/proxy/health_endpoints/test_health_endpoints.py: a request api_key drops the matched deployment's exchange config, client secret, credential name, api_base and vertex_credentials, also with allow_client_side_credentials; a blank api_key still inherits; the endpoint test checks the probe gets the request key and no exchange config. Removing the guard fails all three, and "api_key" in request_params fails the blank-key test
• model management: a team admin patching only client_id or client_secret on an exchange deployment gets 403 and nothing is written; the same patch on an ordinary deployment succeeds
• tests/unit/router_utils/test_cooldown_handlers.py: 401 and 403 on inline and named-credential exchange deployments leave the failure count at 0 and nothing in cooldown; an api_key deployment still cools down on 401, and 429 and 500 on an exchange deployment still cool down; the fallback handler skips 401 and 403 on an exchange deployment and still cools down an api_key deployment
• Dashboard Vitest: selector renders only for providers with auth types, unselected fields are unmounted and absent from the submitted values, required rules follow the selected type, credential edits across types delete stale keys

Mutation checks, each failed its targeted tests and was restored: drop requested_token_use from the OBO form; map 429 to 401; drop oauth_token_exchange_litellm_params from the server-owned set (7 failures); empty RESPONSE_CACHE_EXCLUDED_PROVIDERS; report zero usage; break selector switching; skip stale credential cleanup; drop client_id/client_secret from the write gate; make is_caller_scoped_auth_failure always false, skip its credential lookup, or skip its status check.

Scope: chat completions only (sync, async, fake stream). /v1/messages goes through the Anthropic adapter. /v1/responses is not wired for this provider. Graph's chatOverStream endpoint is not used.

Taxonomy audit

• V1, U2: token_exchange_* are proxy-admin-only server-owned params (request body, model and credential writes); Graph host fixed; api_base ignored
• Response cache: Copilot replies are per caller, so the provider is excluded from response-cache reads and writes
• Rate limits: usage is estimated, never zero, so TPM reservations are not refunded
• M6, M7: cache key covers every config field and the subject token; TTL is expires_in minus 60s and skipped when not positive
• N6, C2: token endpoint 429 and 5xx preserved; Graph statuses pass through
• X1, X4: partial exchange config fails instead of falling back to api_key
• B3: concurrent cold-cache requests for the same user can each exchange once before one is cached. Accepted: exchanges are idempotent and do not change the result
• H4: response = _complete_microsoft_365_copilot(...) in main.py without Final matches every sibling dispatch branch
• H7: the committed _experimental/out bundle is not rebuilt here; bundle refreshes ship separately
• L: Auth Type selection is local state, never a submitted field, and resets on provider change
• F3: sync, async and stream covered; messages and responses out of scope as stated
• Every other item A through BB was checked against the diff and judged not applicable

Type

New Feature

Link to Devin session: https://app.devin.ai/sessions/453c40b054f240f1bf0e4512da9a9aae
Open in Devin Desktop: https://app.devin.ai/desktop/session/453c40b054f240f1bf0e4512da9a9aae?variant=devin
Requested by: @yassin-berriai

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ yassin-berriai
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 90.03322% with 60 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/microsoft_365_copilot/chat/handler.py	86.88%	24 Missing ⚠️
.../llms/microsoft_365_copilot/chat/transformation.py	84.82%	22 Missing ⚠️
litellm/litellm_core_utils/oauth_token_exchange.py	91.66%	11 Missing ⚠️
litellm/llms/microsoft_365_copilot/common_utils.py	93.93%	2 Missing ⚠️
litellm/caching/caching_handler.py	96.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• test_completion_streaming

✅ 31 untouched benchmarks

---

_{Comparing litellm_m365_copilot_obo (70ea230) with main (465f20f)}

[greptile-apps]

Confidence Score: 5/5

[High impact] This revision appears safe to merge; ID-selected connection tests now preserve saved endpoints when replacing a key

Summary

Adds Microsoft 365 Copilot chat with OAuth token exchange, static delegated tokens, and dashboard credential forms. This revision restores saved endpoints when a connection test selects a deployment by model_info.id and supplies a replacement api_key

• LiteLLM can send chat-completion requests to Microsoft 365 Copilot.
• The model form lets admins choose how Microsoft 365 Copilot gets its token.
• Only proxy admins can change the saved token-exchange settings.
• Connection tests use their own credentials and the caller’s headers.
• Copilot replies bypass the shared response cache.
• Caller-specific auth failures no longer cool down an OAuth deployment.

_{Reviews (14) · Last reviewed commit: "fix(proxy): keep saved endpoint when tes..." · Reviewed by Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 3 · PR risk: 0/10

[devin-ai-integration]

@greptileai

[devin-ai-integration]

@greptileai Please review the latest merge from main and confirm the M365, OAuth, health check, router, and dashboard changes

[devin-ai-integration]

@greptileai Please review this CI fix mapping Microsoft 365 Copilot unit tests into the H-to-Z shard

[devin-ai-integration]

@greptileai Please review the latest pushed changes for PR #45158 and refresh the confidence summary against the current head

[devin-ai-integration]

@greptileai Please review commit 2a93a1c and post the updated confidence summary and any findings for PR #45158

[yassin-berriai]

called ms 365 copilot model with OBO token exchange