fix(bedrock): stop emitting Converse cachePoint blocks for Kimi K3

[devin-ai-integration]

TLDR

Problem this solves:

• Converse rejects cachePoint for Kimi K3 ("This model doesn't support the cachePoint field")
• Any cache_control on a request to bedrock/*.moonshotai.kimi-k3 answered HTTP 400
• Clients that always mark prompts cacheable could not use the model at all

How it solves it:

• The three Kimi K3 cost-map rows now say supports_prompt_cache_breakpoint: false
• The Converse cachePoint gate honors that flag before supports_prompt_caching
• supports_prompt_caching stays true, so cached-token pricing is unchanged
• System, message, tool, and toolConfig cache points all go through the same gate
• A deployment's model_info.supports_prompt_cache_breakpoint: false covers an inference profile ARN or a model newer than the map, configured as bedrock/<model> or bedrock/converse/<model>; like every model_info field it applies to every deployment of that model string

User Flow

Before: a developer whose client marks prompts cacheable gets a 400 from every Kimi K3 request

1. The proxy admin adds bedrock/us.moonshotai.kimi-k3 as kimi-k3 to the model list, plus the same model behind an application inference profile ARN as kimi-k3-profile with model_info: {supports_prompt_cache_breakpoint: false}, and starts the proxy
2. The developer sends POST https://litellm-domain/v1/chat/completions with "model": "kimi-k3" (or "kimi-k3-profile") and a system, user, or tool block carrying "cache_control": {"type": "ephemeral"} (or the same marker on an input block of POST https://litellm-domain/v1/responses)
3. They get HTTP 400: BedrockException - {"message":"This model doesn't support the cachePoint field. Remove cachePoint and try again."}
4. Claude Code pointed at the proxy with ANTHROPIC_BASE_URL and ANTHROPIC_MODEL=kimi-k3-profile shows API Error: 400 with that same message on every turn, so they have to strip caching from their client to use the model

After: the same requests answer 200 and the client keeps its caching markers

1. The proxy admin adds bedrock/us.moonshotai.kimi-k3 as kimi-k3 to the model list, plus the same model behind an application inference profile ARN as kimi-k3-profile with model_info: {supports_prompt_cache_breakpoint: false}, and starts the proxy
2. The developer sends POST https://litellm-domain/v1/chat/completions with "model": "kimi-k3" (or "kimi-k3-profile") and a system, user, or tool block carrying "cache_control": {"type": "ephemeral"} (or the same marker on an input block of POST https://litellm-domain/v1/responses)
3. They get HTTP 200 with the model's reply, and usage still reports Bedrock's cached-token counts
4. Claude Code pointed at the proxy with ANTHROPIC_BASE_URL and ANTHROPIC_MODEL=kimi-k3-profile answers every turn, and with ANTHROPIC_MODEL=kimi-k3 it keeps working as it did before

Linear ticket

Resolves LIT-9078

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for both legs, each in its own worktree with its own venv, proxy booted with --num_workers 2 --detailed_debug, the local cost map, and no database, every call reaching real Bedrock in us-east-1 ($PORT is the leg's random port, $LITELLM_MASTER_KEY the proxy master key). The three ARNs are three separate application inference profiles over us.moonshotai.kimi-k3, so each deployment is its own model string:

model_list:
  - model_name: kimi-k3
    litellm_params:
      model: bedrock/us.moonshotai.kimi-k3
      aws_region_name: us-east-1
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
  - model_name: kimi-k3-profile
    litellm_params:
      model: bedrock/arn:aws:bedrock:us-east-1:<account>:application-inference-profile/56nvg84ydzxu
      aws_region_name: us-east-1
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
    model_info:
      supports_prompt_cache_breakpoint: false
  - model_name: kimi-k3-profile-unflagged
    litellm_params:
      model: bedrock/arn:aws:bedrock:us-east-1:<account>:application-inference-profile/xak4zbtb3q6r
      aws_region_name: us-east-1
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
  - model_name: kimi-k3-profile-converse
    litellm_params:
      model: bedrock/converse/arn:aws:bedrock:us-east-1:<account>:application-inference-profile/ak35l6d02pkd
      aws_region_name: us-east-1
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
    model_info:
      supports_prompt_cache_breakpoint: false

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

litellm_settings:
  drop_params: true

Claude Code v2.1.288 is driven in a real tmux session with ANTHROPIC_BASE_URL at the proxy; the same eleven cases run in the same order on both sides, only the proxy commit differs

Before (a292fd4)

A1. Claude Code with ANTHROPIC_MODEL=kimi-k3 (control)

1. Run

~/.claude/bin/claude-tui lit9078c-k3 --cwd ./qa3-before-tui-cwd-k3 --config-dir ./qa3-before-tui-cfg-k3 --env ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT --env ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY --env ANTHROPIC_MODEL=kimi-k3
tmux send-keys -t lit9078c-k3 "Say hi in one word." Enter

2. Observed

Claude Code v2.1.288
kimi-k3 · API Usage Billing
❯ Say hi in one word.
Thought for 2s (ctrl+o to expand)
⏺ Hi!
✻ Cogitated for 3s · done 7:08 PM

A2. Claude Code with ANTHROPIC_MODEL=kimi-k3-profile (the flagged inference profile)

1. Run

~/.claude/bin/claude-tui lit9078c-profile --cwd ./qa3-before-tui-cwd-profile --config-dir ./qa3-before-tui-cfg-profile --env ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT --env ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY --env ANTHROPIC_MODEL=kimi-k3-profile
tmux send-keys -t lit9078c-profile "Say hi in one word." Enter

2. Observed

Claude Code v2.1.288
kimi-k3-profile · API Usage Billing
❯ Say hi in one word.
⏺ API Error: 400 litellm.BadRequestError: BedrockException - {"message":"This model doesn't support the cachePoint field. Remove cachePoint and try again."}
LiteLLM: model group 'kimi-k3-profile' failed with the error above. No fallback was attempted.
✻ Crunched for 2s · done 7:08 PM

B. POST /v1/chat/completions, kimi-k3, cache_control on the system and user blocks

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

C. POST /v1/chat/completions, kimi-k3, cache_control on a tool definition

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get the weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}},"cache_control":{"type":"ephemeral"}}],"tool_choice":"auto"}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

D. POST /v1/messages, kimi-k3, cache_control on the system and user blocks (control)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/messages \
  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -H "x-api-key: $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"system":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}],"messages":[{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"content":[{"type":"text","text":"Hi!"}],"stop_reason":"end_turn","usage":{"input_tokens":106,"output_tokens":25}}
HTTP 200

D2. POST /v1/messages, kimi-k3-profile, the same request

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/messages \
  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -H "x-api-key: $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile","max_tokens":200,"system":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}],"messages":[{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3-profile' failed with the error above. No fallback was attempted."}}
HTTP 400

E. POST /v1/responses, kimi-k3, cache_control on an input block

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/responses \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_output_tokens":200,"input":[{"role":"user","content":[{"type":"input_text","text":"Say hello in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field for user messages. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

F. POST /v1/chat/completions, kimi-k3, no cache_control (control)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse."}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word."}]}]}'

2. Observed

{"choices":[{"finish_reason":"stop","message":{"role":"assistant","content":"Hi."}}],"usage":{"completion_tokens":38,"prompt_tokens":106,"total_tokens":144,"completion_tokens_details":{"reasoning_tokens":21,"text_tokens":17},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":106,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

G. POST /v1/chat/completions, kimi-k3-profile (flagged ARN)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3-profile' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

H. POST /v1/chat/completions, kimi-k3-profile-unflagged (ARN without model_info)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile-unflagged","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3-profile-unflagged' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

I. POST /v1/chat/completions, kimi-k3-profile-converse (bedrock/converse/ ARN, flagged)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile-converse","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3-profile-converse' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

After (5de008b)

A1. Claude Code with ANTHROPIC_MODEL=kimi-k3 (control)

1. Run

~/.claude/bin/claude-tui lit9078d-k3 --cwd ./qa3-after-tui-cwd-k3 --config-dir ./qa3-after-tui-cfg-k3 --env ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT --env ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY --env ANTHROPIC_MODEL=kimi-k3
tmux send-keys -t lit9078d-k3 "Say hi in one word." Enter

2. Observed

Claude Code v2.1.288
kimi-k3 · API Usage Billing
❯ Say hi in one word.
Thought for 2s (ctrl+o to expand)
⏺ Hi!
✻ Baked for 2s · done 7:23 PM

A2. Claude Code with ANTHROPIC_MODEL=kimi-k3-profile (the flagged inference profile)

1. Run

~/.claude/bin/claude-tui lit9078d-profile --cwd ./qa3-after-tui-cwd-profile --config-dir ./qa3-after-tui-cfg-profile --env ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT --env ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY --env ANTHROPIC_MODEL=kimi-k3-profile
tmux send-keys -t lit9078d-profile "Say hi in one word." Enter

2. Observed

Claude Code v2.1.288
kimi-k3-profile · API Usage Billing
❯ Say hi in one word.
Thought for 2s (ctrl+o to expand)
⏺ Hi
✻ Brewed for 2s · done 7:23 PM

B. POST /v1/chat/completions, kimi-k3, cache_control on the system and user blocks

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"chatcmpl-6dca97e3-b5d4-42a4-959d-4360f7c8aedb","created":1790994141,"model":"kimi-k3","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi.","role":"assistant","reasoning_content":"The user wants me to say hi in one word. Terse means brief, concise. One word answer.","thinking_blocks":[{"type":"thinking","thinking":"The user wants me to say hi in one word. Terse means brief, concise. One word answer."}],"provider_specific_fields":{"reasoningContentBlocks":[{"reasoningText":{"text":"The user wants me to say hi in one word. Terse means brief, concise. One word answer."}}]}}}],"usage":{"completion_tokens":39,"prompt_tokens":106,"total_tokens":145,"completion_tokens_details":{"reasoning_tokens":22,"text_tokens":17},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":106,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

C. POST /v1/chat/completions, kimi-k3, cache_control on a tool definition

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get the weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}},"cache_control":{"type":"ephemeral"}}],"tool_choice":"auto"}'

2. Observed

{"id":"chatcmpl-65f90780-4063-40b4-a85b-d355663f24c1","created":1790994142,"model":"kimi-k3","object":"chat.completion","choices":[{"finish_reason":"tool_calls","index":0,"message":{"content":"I'll check the weather in Paris for you.","role":"assistant","tool_calls":[{"index":2,"function":{"arguments":"{\"city\": \"Paris\"}","name":"get_weather"},"id":"call_15ae15b952305d56b17a016e53f5adc2","type":"function"}],"reasoning_content":"The user is asking about the weather in Paris and explicitly wants me to use the tool. Let me call the get_weather function with \"Paris\" as the city.","thinking_blocks":[{"type":"thinking","thinking":"The user is asking about the weather in Paris and explicitly wants me to use the tool. Let me call the get_weather function with \"Paris\" as the city."}],"provider_specific_fields":{"reasoningContentBlocks":[{"reasoningText":{"text":"The user is asking about the weather in Paris and explicitly wants me to use the tool. Let me call the get_weather function with \"Paris\" as the city."}}]}}}],"usage":{"completion_tokens":95,"prompt_tokens":174,"total_tokens":269,"completion_tokens_details":{"reasoning_tokens":33,"text_tokens":62},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":174,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

D. POST /v1/messages, kimi-k3, cache_control on the system and user blocks (control)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/messages \
  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -H "x-api-key: $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"system":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}],"messages":[{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"chatcmpl-f3fed58a-3446-4dc4-8ae1-6c9a6b6f7ff8","type":"message","role":"assistant","model":"kimi-k3","stop_sequence":null,"usage":{"input_tokens":106,"output_tokens":52},"content":[{"type":"thinking","thinking":"The user wants me to say hi in one word. Simple enough. \"Hi\" itself is one word. Or \"Hello.\" Terse instructions mean I keep it minimal.","signature":""},{"type":"text","text":"Hello."}],"stop_reason":"end_turn","stop_details":null}
HTTP 200

D2. POST /v1/messages, kimi-k3-profile, the same request

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/messages \
  -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -H "x-api-key: $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile","max_tokens":200,"system":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}],"messages":[{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"chatcmpl-134395e7-81ac-4ea3-86ce-7eeff57e922d","type":"message","role":"assistant","model":"kimi-k3-profile","stop_sequence":null,"usage":{"input_tokens":106,"output_tokens":39},"content":[{"type":"thinking","thinking":"The user wants me to say hi in one word. Simple request. Just respond with a single word greeting.","signature":""},{"type":"text","text":"Hello!"}],"stop_reason":"end_turn","stop_details":null}
HTTP 200

E. POST /v1/responses, kimi-k3, cache_control on an input block

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/responses \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_output_tokens":200,"input":[{"role":"user","content":[{"type":"input_text","text":"Say hello in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"resp_11Nj1-vnLHsDF8hojsaMfc_Qwf0sOPAPS1x4G8nJar_uE8vnF1PQv7o9BA7tOWSCBZVe0NDGEyUnpHoWaZ0w9htCE0xEzn1pPmsfiFR5WO0KEgPXL_gNChICbwzHIMVjjx5qwEe7O7nT36umnOh5AvhHUaqqmoy_LA2-H9cxgi_RZgPq2Lz8W2UXr8pvivj6O7FK9hRABXcmTY409HuuJJf1sXfxarv7vX9UOWRzSbBhNRwMYXQ6W2H7Wx_UL20WZM4yZlNajMIkjHCf6uAoZyYqLgYfiug4cvGiJvPdnvB55aYh4HExN-gFz7XHUFb9eVrjN3wGh-iL43EwxfLNUhJ1Mn7MI4IfurVDj025lqW2ohahxwM6-RFELxeLL0dtGvdeu7pOeRUayEXb3dRfXgN-Go4D3DmlOc-v2CJSYKmpfQuFTCeSi9q-fzXsPs5pWgP6yJvm","created_at":1790994148,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"kimi-k3","object":"response","output":[{"type":"reasoning","id":"rs_811586cf-a1c2-4aba-8b88-18af4e00f454","status":"completed","role":"assistant","content":[{"type":"output_text","text":"The user asked me to say hello in one word. Simple request. The answer should be just one word.\n\n\"Hello\" itself is one word. Or I could say \"Hi\" or \"Hey\" or \"Greetings.\"\n\nI'll just respond with one word: \"Hello\"","annotations":[]}],"phase":null},{"type":"message","id":"msg_ad61f0b2-79be-45fb-b35d-98b5a83175e2","status":"completed","role":"assistant","content":[{"type":"output_text","text":"Hello","annotations":[]}],"phase":null}],"parallel_tool_calls":false,"temperature":0.0,"tool_choice":"auto","tools":[],"top_p":null,"max_output_tokens":null,"previous_response_id":null,"reasoning":null,"status":"completed","text":{},"truncation":null,"usage":{"input_tokens":91,"input_tokens_details":{"audio_tokens":null,"cached_tokens":0,"cached_tokens_details":null,"image_tokens":null,"text_tokens":91,"video_tokens":null,"cache_write_tokens":0},"output_tokens":72,"output_tokens_details":{"audio_tokens":null,"reasoning_tokens":56,"text_tokens":16},"total_tokens":163,"cost":null},"user":null,"store":null}
HTTP 200

F. POST /v1/chat/completions, kimi-k3, no cache_control (control)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse."}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word."}]}]}'

2. Observed

{"id":"chatcmpl-a00508eb-ec8c-493c-9220-adb3498fc2ba","created":1790994143,"model":"kimi-k3","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi.","role":"assistant","reasoning_content":"The user wants me to say \"hi\" in one word. I'm terse. The answer is simple.","thinking_blocks":[{"type":"thinking","thinking":"The user wants me to say \"hi\" in one word. I'm terse. The answer is simple."}],"provider_specific_fields":{"reasoningContentBlocks":[{"reasoningText":{"text":"The user wants me to say \"hi\" in one word. I'm terse. The answer is simple."}}]}}}],"usage":{"completion_tokens":38,"prompt_tokens":106,"total_tokens":144,"completion_tokens_details":{"reasoning_tokens":22,"text_tokens":16},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":106,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

G. POST /v1/chat/completions, kimi-k3-profile (flagged ARN)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"chatcmpl-6cd13088-401b-453f-9b4c-d993a7e6779b","created":1790994144,"model":"kimi-k3-profile","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi.","role":"assistant","reasoning_content":"The user wants me to say hi in one word. Simple. Just respond with \"Hi\" or \"Hello.\"","thinking_blocks":[{"type":"thinking","thinking":"The user wants me to say hi in one word. Simple. Just respond with \"Hi\" or \"Hello.\""}],"provider_specific_fields":{"reasoningContentBlocks":[{"reasoningText":{"text":"The user wants me to say hi in one word. Simple. Just respond with \"Hi\" or \"Hello.\""}}]}}}],"usage":{"completion_tokens":40,"prompt_tokens":106,"total_tokens":146,"completion_tokens_details":{"reasoning_tokens":23,"text_tokens":17},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":106,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

H. POST /v1/chat/completions, kimi-k3-profile-unflagged (ARN without model_info)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile-unflagged","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"This model doesn't support the cachePoint field. Remove cachePoint and try again.\"}\n\nLiteLLM: model group 'kimi-k3-profile-unflagged' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

I. POST /v1/chat/completions, kimi-k3-profile-converse (bedrock/converse/ ARN, flagged)

1. Run

curl --max-time 150 -s -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"model":"kimi-k3-profile-converse","max_tokens":200,"messages":[{"role":"system","content":[{"type":"text","text":"You are terse.","cache_control":{"type":"ephemeral"}}]},{"role":"user","content":[{"type":"text","text":"Say hi in one word.","cache_control":{"type":"ephemeral"}}]}]}'

2. Observed

{"id":"chatcmpl-4807bffa-360b-4a93-813a-8002904b6902","created":1790994145,"model":"kimi-k3-profile-converse","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi.","role":"assistant","reasoning_content":"The user wants me to say hi in one word. My system prompt says I'm terse. So I should just respond with one word.\n\n\"Hi\" or \"Hello\" or \"Hey\" — any of these work. Keep it to literally one word.","thinking_blocks":[{"type":"thinking","thinking":"The user wants me to say hi in one word. My system prompt says I'm terse. So I should just respond with one word.\n\n\"Hi\" or \"Hello\" or \"Hey\" — any of these work. Keep it to literally one word."}],"provider_specific_fields":{"reasoningContentBlocks":[{"reasoningText":{"text":"The user wants me to say hi in one word. My system prompt says I'm terse. So I should just respond with one word.\n\n\"Hi\" or \"Hello\" or \"Hey\" — any of these work. Keep it to literally one word."}}]}}}],"usage":{"completion_tokens":68,"prompt_tokens":106,"total_tokens":174,"completion_tokens_details":{"reasoning_tokens":52,"text_tokens":16},"prompt_tokens_details":{"cached_tokens":0,"text_tokens":106,"cache_write_tokens":0,"cache_creation_tokens":0},"cache_creation_input_tokens":0,"cache_read_input_tokens":0}}
HTTP 200

Observed alongside the run, from the legs' responses and proxy logs:

• Claude Code with the plain kimi-k3 name answers on both legs: /v1/messages already drops cache markers for a model name it does not recognize as Claude, so the 400 the customer hits needs an ARN deployment (A2, D2), which keeps the markers. Their 1.98.x line predates that drop, so plain names 400 there too
• Before, Claude Code retried the failing turn: four /v1/messages calls for A2, three 400 and one 200. After, two calls, both 200, the same as A1
• kimi-k3-profile-unflagged (H) still 400s on both legs: an ARN carries no model name, so without model_info.supports_prompt_cache_breakpoint: false the gateway has nothing to go on
• /v1/messages and /v1/responses replies carry a thinking block with an empty signature, and /v1/messages usage has no cache_* fields for Kimi K3 (pre-existing on both legs, left alone)
• Claude Code warns that the kimi models are not in its model catalog (client-side, unchanged)

Last updated: tip 744961c, which only adds the audit's integration test module; the proof, /live-pr-risk, and /audit all ran at 5de008b, the last commit with a product diff

Audit round 1 of 3 (head 5de008b, merge base a292fd4, tests commit 744961c)

Every cell is a checked-in integration test in tests/integration/providers/test_bedrock_kimi_k3_cache_point_wire.py, run with tests/integration/run.py providers against a two-worker proxy with real Postgres and Redis; the only double is a local HTTP peer standing in for the Bedrock runtime that records every Converse body it receives and answers the scripted reply. "red" means the cell's expected-on-head assertion fails on the merge base, which is the proof that the cell measures the fix. Each leg ran one cell at a time (--seed 4106601 --order-seed 9078 --workers 1), the head selection twice with the same seeds. The runs: official-base-a292fd409f-20261003T040408Z (38 red, 27 green, the exact expected shape), official-head1-5de008b605-20261003T040856Z (64 green, S10/S11 red once, see below), official-head2-5de008b605-20261003T041758Z (65 green). The first head run's one red was the _model_id test helper, not the product: it looked a just-created deployment up through GET /model/info, which answers from the worker-local router (model_info_v1 reads llm_router.model_list), while a second uvicorn worker only learns a /model/new row on its next DB sync (add_deployment_job, or the Redis config-sync signal), and the test client's keep-alive expires after 2 s idle (GATEWAY_LIMITS), so the lookup rode a fresh connection to the worker that had not synced yet. The helper now reads the id from the LiteLLM_ProxyModelTable row that /model/new wrote, and per the tests-only rerun rule the four cells that use it (S10/S11, X1, X6, C3) reran at the same tip and seeds as rerun-base-a292fd409f-20261003T043420Z (3 red, 1 green, as expected), rerun-head1-5de008b605-20261003T043622Z and rerun-head2-5de008b605-20261003T043946Z; those rows carry the rerun result (run column), every other row keeps its official result

| inv | endpoint | mode | client | scenario | node id | base | head x2 | run | verdict |
| M1 | chat | plain | httpx | YAML bedrock/us.moonshotai.kimi-k3, 3 cache_control | test_m01_to_m05_every_kimi_k3_form_sends_converse_without_a_cache_point[m-kimi-us] | red | green green | official | PASS |
| M2 | chat | plain | httpx | YAML bedrock/converse/global.moonshotai.kimi-k3 | test_m01_to_m05_every_kimi_k3_form_sends_converse_without_a_cache_point[m-kimi-global-converse] | red | green green | official | PASS |
| M3 | chat | plain | httpx | YAML bedrock/moonshotai.kimi-k3 | test_m01_to_m05_every_kimi_k3_form_sends_converse_without_a_cache_point[m-kimi-base] | red | green green | official | PASS |
| M4 | chat | plain | httpx | YAML bedrock/us-east-1/us.moonshotai.kimi-k3 | test_m01_to_m05_every_kimi_k3_form_sends_converse_without_a_cache_point[m-kimi-regional] | red | green green | official | PASS |
| M15 | chat | plain | httpx | YAML ARN, model_info flag false | test_m01_to_m05_every_kimi_k3_form_sends_converse_without_a_cache_point[m-arn-yaml-flagged] | red | green green | official | PASS |
| M9 | chat | plain | httpx | Claude control keeps 3 cachePoints | test_m08_m09_models_that_take_cache_points_still_get_all_three[m-claude-control] | green | green green | official | PASS |
| M16 | chat | plain | httpx | YAML ARN without model_info keeps 3 | test_m08_m09_models_that_take_cache_points_still_get_all_three[m-arn-yaml-plain] | green | green green | official | PASS |
| M10 | chat | plain | httpx | gpt-oss control, never had cachePoints | test_m10_m11_models_without_caching_never_got_cache_points[m-gpt-oss-control] | green | green green | official | PASS |
| M11 | chat | plain | httpx | llama4 control, never had cachePoints | test_m10_m11_models_without_caching_never_got_cache_points[m-llama-control] | green | green green | official | PASS |
| M14 | chat | plain | httpx | wildcard bedrock/*, Kimi strips, Claude keeps | test_m14_a_wildcard_deployment_resolves_the_caller_model_before_deciding | red | green green | official | PASS |
| E1 | chat | plain | httpx | Kimi, cache_control on system, user, tool | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-chat-plain] | red | green green | official | PASS |
| E7 | chat | stream | httpx | Kimi, same markers | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-chat-stream] | red | green green | official | PASS |
| E8 | messages | plain | httpx | Kimi, adapter drops cache_control on both legs | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-messages-plain] | green | green green | official | PASS |
| E8 | messages | stream | httpx | Kimi, same adapter rule | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-messages-stream] | green | green green | official | PASS |
| E9 | responses | plain | httpx | Kimi, cache_control on input block | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-responses-plain] | red | green green | official | PASS |
| E6 | responses | stream | httpx | Kimi, same | test_e01_to_e06_every_endpoint_reaches_kimi_without_a_cache_point[e-responses-stream] | red | green green | official | PASS |
| E10 | chat | plain | httpx | Claude control keeps 2 cachePoints | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-chat-plain] | green | green green | official | PASS |
| E10 | chat | stream | httpx | Claude control keeps 2 | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-chat-stream] | green | green green | official | PASS |
| E10a | messages | plain | httpx | Claude rides Invoke, 2 cache_control survive | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-messages-plain] | green | green green | official | PASS |
| E10a | messages | stream | httpx | Claude rides Invoke stream, 2 survive | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-messages-stream] | green | green green | official | PASS |
| E10 | responses | plain | httpx | Claude control keeps 1 | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-responses-plain] | green | green green | official | PASS |
| E10 | responses | stream | httpx | Claude control keeps 1 | test_e07_to_e12_every_endpoint_still_sends_claude_its_cache_points[e-responses-stream] | green | green green | official | PASS |
| E12 | messages | plain | httpx | flagged ARN strips, plain ARN keeps 2 | test_e13_e14_messages_keeps_cache_control_for_an_arn_and_the_flag_decides[e-messages-arn-plain] | red | green green | official | PASS |
| E12 | messages | stream | httpx | same pair, streaming | test_e13_e14_messages_keeps_cache_control_for_an_arn_and_the_flag_decides[e-messages-arn-stream] | red | green green | official | PASS |
| E1 | chat | plain | OpenAI SDK sync | Kimi strips | test_k01_openai_sdk_sync_chat_reaches_kimi_without_a_cache_point | red | green green | official | PASS |
| E2 | chat | stream | OpenAI SDK async | Kimi strips | test_k02_openai_sdk_async_chat_stream_reaches_kimi_without_a_cache_point | red | green green | official | PASS |
| E3 | messages | plain | Anthropic SDK sync | Kimi, adapter rule, both legs strip | test_k03_anthropic_sdk_sync_messages_reaches_the_model_without_a_cache_point[k03-kimi] | green | green green | official | PASS |
| E11 | messages | plain | Anthropic SDK sync | flagged ARN strips | test_k03_anthropic_sdk_sync_messages_reaches_the_model_without_a_cache_point[k03-flagged-arn] | red | green green | official | PASS |
| E4 | messages | stream | Anthropic SDK async | Kimi, adapter rule, both legs strip | test_k04_anthropic_sdk_async_stream_reaches_the_model_without_a_cache_point[k04-kimi] | green | green green | official | PASS |
| E11 | messages | stream | Anthropic SDK async | flagged ARN strips | test_k04_anthropic_sdk_async_stream_reaches_the_model_without_a_cache_point[k04-flagged-arn] | red | green green | official | PASS |
| E5 | responses | plain | OpenAI SDK sync | Kimi strips | test_k05_openai_sdk_sync_responses_reaches_kimi_without_a_cache_point | red | green green | official | PASS |
| E6 | responses | stream | OpenAI SDK async | Kimi strips | test_k06_openai_sdk_async_responses_stream_reaches_kimi_without_a_cache_point | red | green green | official | PASS |
| L1 | chat | plain | httpx | system marker only: Kimi 0, Claude 1 | test_l01_to_l03_each_marker_location_is_dropped_for_kimi_and_kept_for_claude[l-system] | red | green green | official | PASS |
| L2 | chat | plain | httpx | user marker only: Kimi 0, Claude 1 | test_l01_to_l03_each_marker_location_is_dropped_for_kimi_and_kept_for_claude[l-user] | red | green green | official | PASS |
| L3 | chat | plain | httpx | tool marker only: Kimi 0, Claude 1 | test_l01_to_l03_each_marker_location_is_dropped_for_kimi_and_kept_for_claude[l-tool] | red | green green | official | PASS |
| L4 L5 L6 | chat | plain | httpx | YAML injection points: Kimi 0, Claude 1 at system[-1] | test_l04_l05_gateway_injection_points_are_dropped_for_kimi_and_kept_for_claude | red | green green | official | PASS |
| L7 | chat | plain | httpx | no markers, body untouched | test_l06_a_request_without_markers_was_never_touched | green | green green | official | PASS |
| P1 | chat | plain | httpx | cache read 100, write 50: usage and cost from the Kimi row | test_p01_kimi_cache_usage_is_still_priced_from_its_own_row | green | green green | official | PASS |
| P2 | chat | plain | httpx | plain Kimi reply priced from its row | test_p02_a_plain_kimi_reply_is_priced_from_its_own_row | green | green green | official | PASS |
| S7 | chat | plain | httpx | 5 KB caller model: 4xx before the peer, liveliness 200 | test_s07_a_5kb_caller_model_under_the_wildcard_is_a_bounded_4xx_and_liveliness_stays_up | green | green green | official | PASS |
| S8 | chat | plain | httpx | peer 400 validationException reaches the caller | test_s08_a_peer_rejection_on_kimi_is_a_400_with_the_message_and_a_failure_row[s08-plain] | green | green green | official | PASS |
| S8 | chat | stream | httpx | same, streaming | test_s08_a_peer_rejection_on_kimi_is_a_400_with_the_message_and_a_failure_row[s08-stream] | green | green green | official | PASS |
| S9 | chat | plain | httpx | bogus key: 401, no peer call | test_s09_an_unauthenticated_kimi_request_never_reaches_the_peer | green | green green | official | PASS |
| X2 | chat | plain | httpx | two no-cache sends: two peer calls, two rows | test_x02_two_identical_uncached_requests_are_two_peer_calls_and_two_rows | green | green green | official | PASS |
| X3 | chat | plain | httpx | response cache hit answers with fewer peer calls | test_x03_a_response_cache_hit_still_answers_after_fewer_peer_calls_than_sends | green | green green | official | PASS |
| X5 | chat | plain | httpx | ARN with openai. segment, pre-existing family rule | test_x05_an_arn_whose_profile_name_starts_with_openai_dot_is_classified_by_the_pre_existing_family_rule | green | green green | official | PASS |
| M5 | chat messages responses | plain | httpx | /model/new bedrock/<arn> flag false | test_m05_to_m07_a_deployment_flag_false_on_an_arn_strips_cache_points_on_every_route[m05-plain-route] | red | green green | official | PASS |
| M6 | chat messages responses | plain | httpx | /model/new bedrock/converse/<arn> flag false | test_m05_to_m07_a_deployment_flag_false_on_an_arn_strips_cache_points_on_every_route[m06-converse-route] | red | green green | official | PASS |
| M7 | chat messages responses | plain | httpx | /model/new bedrock/converse_like/<arn> flag false | test_m05_to_m07_a_deployment_flag_false_on_an_arn_strips_cache_points_on_every_route[m07-converse-like-route] | red | green green | official | PASS |
| M8 | chat | plain | httpx | /model/new ARN without flag keeps 3 | test_m08_an_arn_without_the_flag_still_gets_cache_points | green | green green | official | PASS |
| M12 | chat | plain | httpx | /model/new Kimi flag true overrides the row | test_m12_a_deployment_flag_true_on_kimi_overrides_the_cost_map_row | green | green green | official | PASS |
| M13 X4 | chat | plain | httpx | /model/new Kimi flag null falls back to the row | test_m13_a_null_deployment_flag_on_kimi_falls_back_to_the_cost_map_row | red | green green | official | PASS |
| S1 | chat | plain | httpx | flag "true" read as false | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s01-string-true] | red | green green | official | PASS |
| S2 | chat | plain | httpx | flag 1 read as false | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s02-int-one] | red | green green | official | PASS |
| S3 | chat | plain | httpx | flag 0 | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s03-int-zero] | red | green green | official | PASS |
| S4 | chat | plain | httpx | flag [] | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s04-empty-list] | red | green green | official | PASS |
| S5 | chat | plain | httpx | flag "" | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s05-empty-string] | red | green green | official | PASS |
| S6 | chat | plain | httpx | flag 5 KB string | test_s01_to_s06_an_odd_typed_flag_is_read_as_false_never_as_true[s06-5kb-string] | red | green green | official | PASS |
| S10 S11 | chat | plain | httpx | two deployments of one ARN share the flag, delete keeps it | test_s10_s11_two_deployments_of_one_arn_share_the_flag_and_a_delete_leaves_it_in_place | red | green green | rerun | PASS |
| X1 | chat | plain | httpx | PATCH /model/{id}/update flips the flag to true | test_x01_flipping_the_flag_to_true_through_a_patch_update_turns_cache_points_back_on | red | green green | rerun | PASS |
| X6 | chat | plain | httpx | legacy POST /model/update: 200, stored flag unchanged, body unchanged | test_x06_the_legacy_post_update_answers_200_and_leaves_the_stored_flag_alone | green | green green | rerun | PASS |
| C1 | chat messages responses | mixed | httpx async | peer outage mid burst, recovery lands every id once | test_c01_a_peer_outage_mid_traffic_fails_cleanly_and_recovery_lands_every_id_once | red | green green | official | PASS |
| C2 | chat | stream | httpx async | SIGKILL one worker, survivor and respawn strip | test_c02_a_worker_killed_mid_traffic_leaves_the_survivor_and_the_respawn_stripping | red | green green | official | PASS |
| C3 | chat | plain | httpx | proxy restart re-registers the stored flag | test_c03_a_restart_re_registers_a_stored_deployment_flag_from_the_database | red | green green | rerun | PASS |
| C4 | chat | stream | httpx async | slow peer, 10 concurrent streams, rows once | test_c04_a_slow_peer_under_ten_concurrent_streams_completes_every_call_once | red | green green | official | PASS |

Live rows V1 and V2 (real Bedrock answers the Kimi K3 request without cachePoint and rejects it with one, for the model id and for an inference profile ARN) are the QA run above at the same two commits and are not repeated here

Findings from the audit, none introduced by this PR: the legacy POST /model/update answers 200 and never writes model_info, so a supports_prompt_cache_breakpoint flip through it does not land (the Admin UI uses PATCH /model/{id}/update, which does); a truthy non-boolean flag value such as "true" or 1 is read as false by the gate's is True check because /model/new does not validate the field's type

Type

🐛 Bug Fix

Caveats (if any)

Low

• A deployment's supports_prompt_cache_breakpoint lands in the model's shared cost-map row, like every model_info field, so it governs every deployment of that model string (a mapped model's built-in row included) and /model/delete of that deployment does not withdraw it until a restart or cost-map reload. The same scope already applied to the supports_prompt_caching read this gate had before the PR, and the capability is a property of the model, not of a deployment, so a deployment-level read (which would need a deployment handle the Converse transformation does not have) is net-negative to add; set the flag on a model string whose deployments all need it
• The customer's 1.98.x line has no Converse cachePoint gate to pick this onto (the gate first shipped in v1.103.0), so there is no clean backport and the fix reaches them on upgrade; the workaround on that line is stripping cache_control client-side (DISABLE_PROMPT_CACHING=1 in Claude Code) or in a pre-call hook
• osv-scan is red on braces 3.0.3 (GHSA-vfj7-8cjw-p6xm, dashboard dev dependency, no fix published); not from this PR, main's run 36975463472 was green on the same lockfile, and it is not a required check
• cache_control on a Kimi K3 request is now dropped silently rather than answered 400, the same rule this gate already applies to OpenAI-family models on Bedrock: the marker is a translation the gateway emits, not a caller parameter, so a 400 for it would be the gateway failing its own request
• An inference profile ARN is not in the cost map, so it still emits cache points unless a deployment sets model_info.supports_prompt_cache_breakpoint: false; flipping that default would silently turn caching off for every ARN deployment of a model that accepts cache points, which is the invariant the gate protects
• Bedrock's native chat-completions endpoint accepts explicit caching for Kimi K3; routing there is a separate feature (feat(bedrock): pass chat through to each model's native openai endpoint on runtime and mantle #43581), out of scope here
• A bedrock/invoke/<model> deployment of a model already in the cost map registers its flag on the built-in row, which the gate does not reach from the invoke/ form, so the flag is ignored there; only Nova runs the invoke route through Converse and Nova accepts cache points, so no model hits this today, and the bedrock/<model> route reads it
• The gate reads supports_prompt_cache_breakpoint as a strict boolean: a string "true" sent through /model/new counts as false, since the router's model_info takes extra fields without validating their type (YAML true/false parse as booleans, so config files are unaffected); type validation belongs to the /model/new schema, not this gate

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
• 5de008b passes /live-pr-risk
• 744961c passes /audit

---

Note

Medium Risk
Changes Bedrock Converse request shaping for all models that use the cache-point gate; behavior shifts for Kimi K3 and any deployment overriding the new flag, though Claude and unmapped models keep prior semantics.

Overview
Fixes 400 errors when clients send cache_control to Bedrock Kimi K3 on Converse: the model can bill cached tokens but rejects explicit cachePoint blocks.

bedrock_model_accepts_cache_points now honors a new cost-map / deployment flag supports_prompt_cache_breakpoint (strict boolean) before supports_prompt_caching, and looks up router-registered keys (bedrock/, bedrock/converse/, bedrock/converse_like/) so inference-profile ARNs can opt out via model_info. Kimi K3 rows are set to supports_prompt_cache_breakpoint: false while supports_prompt_caching stays true so pricing of cache read/write tokens is unchanged.

Adds broad unit coverage and a large proxy integration suite that asserts Converse bodies omit cachePoint for Kimi (and still emit them for Claude) across chat, messages, responses, SDKs, and deployment flag edge cases.

^{Reviewed by Cursor Bugbot for commit 744961c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds a new flag to control cache point emission for specific models.

The PR appears safe to merge; no new actionable issue or outstanding previous finding was identified.

Summary

The PR prevents Bedrock Converse from sending cachePoint blocks to Kimi K3 while retaining cached-token pricing.

• Adds a capability flag to the Kimi K3 model-price entries and honors deployment overrides, including inference-profile ARN routes.
• Adds unit and integration coverage for cache markers across request surfaces and for pricing behavior.

_{Reviews (6) · Last reviewed commit: "test(bedrock): add the Kimi K3 cache poi..."}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_kimi_k3_no_cachepoint (744961c) with main (8efb4a2)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 744961c. Configure here.}

[mateo-berri]

LGTM