fix(vertex-ai): reuse anthropic messages config instances

[Sameerlite]

Summary

• cache ProviderConfigManager.get_provider_anthropic_messages_config() via an existing @lru_cache path
• ensure repeated Vertex Anthropic messages lookups reuse the same config instance so VertexBase credential cache is preserved across calls
• add a regression test that fails if config-instance reuse is broken in future changes

Test plan

• pytest tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_messages_config.py::test_provider_config_manager_reuses_vertex_anthropic_messages_config_instance -v

[greptile-apps]

Greptile Summary

This PR wraps ProviderConfigManager.get_provider_anthropic_messages_config in an @lru_cache-backed private helper so that repeated lookups for the same (model, provider) pair return the same object, keeping VertexBase._credentials_project_mapping intact across calls. A regression test guards the object-identity guarantee with proper cache_clear bookkeeping.

Confidence Score: 5/5

Safe to merge — the change is additive and strictly improves credential-reuse over the previous always-new-instance behaviour.

No correctness regressions found. The only concern (cache eviction silently discarding the credential state) is a pre-existing limitation of LRU and represents strictly better behaviour than before. All remaining observations are P2.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/utils.py	Introduces a private _get_provider_anthropic_messages_config_cached static method decorated with @lru_cache so repeated lookups for the same (model, provider) pair return the same config instance, preserving the VertexBase credential cache. Side effect: all other providers (Anthropic, Bedrock, Azure AI, Minimax) also become singletons per cache entry, and eviction under the default 64-entry cap can silently reset the state this PR aims to preserve.
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_messages_config.py	Adds a focused regression test that clears and restores the LRU cache in a try/finally guard, confirms object identity for two consecutive lookups of the same Vertex Claude model, and validates isinstance against the expected concrete type. Clean and correctly isolated.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant get_provider_anthropic_messages_config
    participant lru_cache
    participant _get_provider_anthropic_messages_config_cached

    Caller->>get_provider_anthropic_messages_config: (model, provider)
    get_provider_anthropic_messages_config->>lru_cache: lookup (model, provider)
    alt Cache HIT
        lru_cache-->>get_provider_anthropic_messages_config: existing config instance
    else Cache MISS
        lru_cache->>_get_provider_anthropic_messages_config_cached: (model, provider)
        _get_provider_anthropic_messages_config_cached-->>lru_cache: new VertexAIPartnerModelsAnthropicMessagesConfig()
        lru_cache-->>get_provider_anthropic_messages_config: new config instance (stored)
    end
    get_provider_anthropic_messages_config-->>Caller: config (same instance on cache HIT)

Loading

_{Reviews (2): Last reviewed commit: "fix(vertex-ai): reuse anthropic messages..." | Re-trigger Greptile}

[greptile-apps]

All providers now return singleton config instances

The @lru_cache is applied to the entire dispatch function, so AnthropicMessagesConfig, AzureAnthropicMessagesConfig, MinimaxMessagesConfig, and the Bedrock config are also now singletons — not just the Vertex AI path. The PR description focuses only on preserving the VertexBase credential cache, but the caching is broader than that. This is harmless today because the other config classes appear stateless, but if any of them acquires mutable per-call instance state in the future it will silently be shared across requests. A more targeted alternative would cache only for VERTEX_AI or move the credential cache into a class-level dict on VertexBase itself.

[yuneng-berri]

@Sameerlite Please see if this is applicable. This sounds like a bug waiting to happen

[veria-ai]

Low: No security issues found

This PR adds lru_cache to a provider config lookup to reuse config instances. The cache key is (model, provider) and the returned objects are stateless transformation configs. No security concerns.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-24T03:50:34.909Z