fix(guardrails): block private destinations in custom code http_request and bound guardrail execution time

[devin-ai-integration]

TLDR

Problem this solves:

• http_request in a custom code guardrail could reach loopback, private and cloud metadata addresses
• Guardrail code had no execution limit, so one busy loop stalled every request on the proxy
• The admin "test custom code" endpoint hung forever on an infinite loop
• A guardrail whose code failed to compile was still saved to the DB by POST /guardrails

How it solves it:

• http_request, http_get, http_post go through the shared SSRF validator, every redirect hop included
• litellm_settings.user_url_allowed_hosts opts an internal host back in, user_url_validation: false turns the check off
• New timeout on the guardrail (default 30s) bounds each run, module-level code included
• Every while test, for iteration and comprehension in guardrail code checks the budget, so a loop that never yields or awaits still stops
• Sync guardrail code runs on a worker thread, so the event loop keeps serving while it runs
• A run that finishes after its deadline is reported as a timeout, even if the code swallowed the interrupt
• The test endpoint returns "Execution timeout" promptly and awaits async guardrails instead of returning a coroutine
• A compile failure at POST /guardrails now rolls the DB write back and returns 400, and any guardrail type whose configuration fails validation there now gets a 400 instead of a 500 wrapping it
• A caller-supplied Host header on http_* is dropped when validation is on, so only the validated host is sent

Intentional product change: a guardrail that called a private or loopback address now gets {"success": false, "error": "Blocked URL: ..."} from the primitive until the host is allowlisted, and a run longer than 30s now fails the request

User Flow

Before: a proxy admin's custom code guardrail can read an internal-only service on the proxy host, and a busy loop in another guardrail stalls every request on the proxy

1. The admin configures a custom_code guardrail whose apply_guardrail calls await http_get("http://127.0.0.1:40023/internal.txt") and starts the proxy
2. A developer sends POST http://litellm-domain/v1/chat/completions with "guardrails": ["loopback-probe"]
3. The reply is 200 with finish_reason: content_filter, and the guardrail's message carries the internal service's body, which the service logged as a hit
4. A second guardrail whose code is while n < 550000000: n += 1 with timeout: 5 takes 22s to answer, and GET http://litellm-domain/health/liveliness sent 3s into it waits 17s
5. Anyone who can edit guardrails can read whatever the proxy host can reach, a cloud metadata endpoint included

After: the same loopback call is refused unless the host is allowlisted, and the busy loop fails its own request at 5s while the proxy keeps answering

1. The admin configures a custom_code guardrail whose apply_guardrail calls await http_get("http://127.0.0.1:40023/internal.txt") and starts the proxy
2. A developer sends POST http://litellm-domain/v1/chat/completions with "guardrails": ["loopback-probe"]
3. The reply is 200 with finish_reason: content_filter, and the message reads Blocked URL: URL targets a blocked address (127.0.0.1); the internal service logs no hit, and litellm_settings.user_url_allowed_hosts: ["127.0.0.1:40023"] lets that one host through again
4. The busy-loop guardrail's request fails at 5s with exceeded its 5s execution timeout, GET http://litellm-domain/health/liveliness sent meanwhile answers in milliseconds, and an async while True: pass guardrail fails the same way
5. Anyone who can edit guardrails can only reach hosts the proxy admin allowlisted

Relevant issues

Supersedes #36983, #32870 and #32897, which covered the SSRF half without redirect validation or the execution limit

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Before (4adbc13)

Proxy booted from 4adbc13, proxy on :41917 with --num_workers 2, real OpenAI calls (gpt-6-luna), curl as the client

Control: no guardrail

$ curl -s http://127.0.0.1:41917/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Say hi in three words."}]}'
{"id":"chatcmpl-ESDwJhz1FOviIfBGXQXFqLzWt9zQV","created":1790395475,"model":"gpt-6-luna","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi there, friend.","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{}}],"usage":{"completion_tokens":25,"prompt_tokens":12,"total_tokens":37,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":11,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0,"cache_write_tokens":0,"cache_creation_tokens":0}},"service_tier":"default"}
-> http 200 in 16.213023s

Case 1: guardrail http_get to a loopback address, on every unified endpoint

$ curl -s http://127.0.0.1:41917/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["loopback-probe"]}'
{"id":"chatcmpl-af8c1813-b374-4d18-bd53-e73c72082337","created":1790395476,"model":"gpt-6-luna","object":"chat.completion","system_fingerprint":null,"choices":[{"finish_reason":"content_filter","index":0,"message":{"content":"guardrail reached loopback: HTTP 200 body=internal-only marker: this loopback service is not reachable from the internet\n","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields":null}}],"usage":{"completion_tokens":0,"prompt_tokens":0,"total_tokens":0,"completion_tokens_details":null,"prompt_tokens_details":null}}
-> http 200 in 0.061830s

$ curl -s http://127.0.0.1:41917/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"guardrails":["loopback-probe"]}'
{"id":"msg_78a99bc4-8ab4-495e-9cc2-97d4e9a780cb","type":"message","role":"assistant","content":[{"type":"text","text":"guardrail reached loopback: HTTP 200 body=internal-only marker: this loopback service is not reachable from the internet\n"}],"model":"gpt-6-luna","stop_reason":"end_turn","usage":{"input_tokens":0,"output_tokens":0}}
-> http 200 in 0.055958s

$ curl -s http://127.0.0.1:41917/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","input":"hi","guardrails":["loopback-probe"]}'
{"id":"resp_8f683fb0-c281-4fa6-8721-3ed1f8472437","created_at":1790395476,"error":null,"incomplete_details":null,"instructions":null,"metadata":null,"model":"gpt-6-luna","object":"response","output":[{"content":[{"type":"text","text":"guardrail reached loopback: HTTP 200 body=internal-only marker: this loopback service is not reachable from the internet\n"}]}],"parallel_tool_calls":null,"temperature":null,"tool_choice":null,"tools":null,"top_p":null,"max_output_tokens":null,"previous_response_id":null,"reasoning":null,"status":"completed","text":null,"truncation":null,"usage":{"input_tokens":0,"input_tokens_details":null,"output_tokens":0,"output_tokens_details":null,"total_tokens":0,"cost":null},"user":null,"store":null}
-> http 200 in 0.048169s

$ # loopback service access log: hits during the three calls above
3 hits
  127.0.0.1 - - [25/Sep/2026 21:04:36] "GET /internal.txt HTTP/1.1" 200 -
  127.0.0.1 - - [25/Sep/2026 21:04:36] "GET /internal.txt HTTP/1.1" 200 -
  127.0.0.1 - - [25/Sep/2026 21:04:36] "GET /internal.txt HTTP/1.1" 200 -

Case 2: sync busy loop with timeout: 5, two concurrent requests (one per worker) and a liveliness probe 3s in

$ for i in 1 2; do curl -s http://127.0.0.1:41917/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["busy-loop"]}' -o busy_$i.json -w "busy $i: http %{http_code} in %{time_total}s\n" & sleep 0.5; done
$ sleep 3; curl -s http://127.0.0.1:41917/health/liveliness -w " <- liveliness: http %{http_code} in %{time_total}s\n"; wait; cat busy_1.json
"I'm alive!" <- liveliness: http 200 in 19.425880s
busy 1: http 200 in 24.259038s
busy 2: http 200 in 24.939655s
{"id":"chatcmpl-ESDwiLyhWrD0eMZ10nHQ8H8RJ3Yr7","created":1790395500,"model":"gpt-6-luna","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi! How can I help you today?","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{}}],"usage":{"completion_tokens":27,"prompt_tokens":7,"total_tokens":34,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":9,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0,"cache_write_tokens":0,"cache_creation_tokens":0}},"service_tier":"default"}

Case 3: the same busy loop on /v1/messages and /v1/responses

$ curl -s http://127.0.0.1:41917/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"guardrails":["busy-loop"]}'
{"id":"resp_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOm9wZW5haTttb2RlbF9pZDo5MzQzYThjY2I3OGVhZjVjNGE5MGIxN2M3ZTg1NDI1MDc2MzFjNTkxYTQ1ZjRhNjE4MWEwMmM1NDJkNGI4NDY3O3Jlc3BvbnNlX2lkOnJlc3BfMDM4NGIzYTY4MTcxNTMyMTAwNmFiNzQ0ODJjYjRjODdkMDlhYzI3YzA2YjE2M2JmZGQ=","type":"message","role":"assistant","model":"gpt-6-luna","stop_sequence":null,"usage":{"input_tokens":7,"output_tokens":13},"content":[{"type":"text","text":"Hi! How can I help you today?"}],"stop_reason":"end_turn","stop_details":null}
-> http 200 in 21.743134s

$ curl -s http://127.0.0.1:41917/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","input":"hi","guardrails":["busy-loop"]}'
{"id":"resp_ZXqkqssODLxxRp17YxWzserjs1fS5IAegTtu4j_a6GH4dlMSVfsKv1q_cT66rgBY9wpvlvSFt_7tVjk7hWEiMuxtqhYzB0YaMIYpOKTnfeWeA6zKgn9LnWbKwWewcSIHvO2Q8tTmZYcpjhXjvBucMrNybMJXHjW3bjgQOdFDKeE5HfHaJNgN54iYyoS19KojZK0nmRVmjv-UFG5wVRwHC8yn6LQcX9On8WoN7_5X5zOYgdIwqQj2qhwhdBIoLfFqAVNeA3y6Prfw8X7m_EfXtnURovRHJJAjCz_N9ogg0xpLhuMYLyS57_2szlrApasz6WriPncpJyw8x8iPGbEh42n41nwLC1MOMbrE9L4xhczMiWM0hQGVTaEbdlbaWW8FW4WhbCuY0tFRO1i0W44UvczMN7TjGXjMPAiSbhTXQx9szxzgPHxQsYez9JnbxgKqL7qAhmSnDufk-9_4Pmrl1DFB","created_at":1790395545,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"gpt-6-luna","object":"response","output":[{"id":"msg_029b5986c7d57e04006ab7449a1d5c87d0a7c673033bbc872a","content":[{"annotations":[],"text":"Hi! How can I help?","type":"output_text","logprobs":[]}],"role":"assistant","status":"completed","type":"message","phase":"final_answer"}],"parallel_tool_calls":true,"temperature":1.0,"tool_choice":"auto","tools":[],"top_p":0.98,"max_output_tokens":null,"previous_response_id":null,"reasoning":{"context":"all_turns","effort":"medium","mode":"standard","summary":null},"status":"completed","text":{"format":{"type":"text"},"verbosity":"medium"},"truncation":"disabled","usage":{"input_tokens":7,"input_tokens_details":{"audio_tokens":null,"cached_tokens":0,"cached_tokens_details":null,"image_tokens":null,"text_tokens":null,"video_tokens":null,"cache_write_tokens":0},"output_tokens":11,"output_tokens_details":{"audio_tokens":null,"reasoning_tokens":0,"text_tokens":null},"total_tokens":18,"cost":null},"user":null,"store":true,"access_programs":{"cyber":"daybreak_blue"},"background":false,"billing":{"payer":"developer"},"completed_at":1790395546,"frequency_penalty":0.0,"max_tool_calls":null,"moderation":null,"presence_penalty":0.0,"prompt_cache_key":null,"prompt_cache_retention":"24h","safety_identifier":null,"service_tier":"default","tool_usage":{"image_gen":{"input_tokens":0,"input_tokens_details":{"image_tokens":0,"text_tokens":0},"output_tokens":0,"output_tokens_details":{"image_tokens":0,"text_tokens":0},"total_tokens":0},"web_search":{"num_requests":0}},"top_logprobs":0}
-> http 200 in 22.799320s

Case 4: admin test endpoint with an infinite loop (curl gives up after 20s)

$ curl -s --max-time 20 -X POST http://127.0.0.1:41917/guardrails/test_custom_code -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"custom_code":"def apply_guardrail(inputs, request_data, input_type):\n    while True:\n        pass","test_input":{"texts":["hi"]},"input_type":"request"}'; echo "curl exit=$?"

-> http 000 in 20.007649s
curl exit=28

Case 5: async guardrail that loops without ever awaiting, timeout: 5 (curl gives up after 30s)

$ curl -s --max-time 30 http://127.0.0.1:41917/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["async-busy-loop"]}'; echo "curl exit=$?"

-> http 000 in 30.001686s
curl exit=28

Case 6: is the proxy still alive after cases 4 and 5?

$ curl -s --max-time 5 http://127.0.0.1:41917/health/liveliness -w " <- liveliness: http %{http_code} in %{time_total}s\n"; echo "curl exit=$?"
 <- liveliness: http 000 in 5.009352s
curl exit=28

After (649e590)

Commit since: 5372792 adds tests only (git diff 649e590f9e..5372792aa7 -- litellm is empty), so this run at 649e590 stands

Proxy booted from 649e590, proxy on :35334 with --num_workers 2, real OpenAI calls (gpt-6-luna), curl as the client

Control: no guardrail

$ curl -s http://127.0.0.1:35334/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"Say hi in three words."}]}'
{"id":"chatcmpl-ESEUfjogJtHya0DsIJepiQvp6mW5U","created":1790397605,"model":"gpt-6-luna","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi there, friend.","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{}}],"usage":{"completion_tokens":32,"prompt_tokens":12,"total_tokens":44,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":18,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0,"cache_write_tokens":0,"cache_creation_tokens":0}},"service_tier":"default"}
-> http 200 in 1.288322s

Case 1: guardrail http_get to a loopback address, on every unified endpoint

$ curl -s http://127.0.0.1:35334/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["loopback-probe"]}'
{"id":"chatcmpl-32dca52a-4be8-4d45-8778-b3b8ac9bad92","created":1790397605,"model":"gpt-6-luna","object":"chat.completion","system_fingerprint":null,"choices":[{"finish_reason":"content_filter","index":0,"message":{"content":"guardrail could not reach loopback: Blocked URL: URL targets a blocked address (127.0.0.1). If this is a legitimate internal service, add the host to `user_url_allowed_hosts` in litellm_settings.","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields":null}}],"usage":{"completion_tokens":0,"prompt_tokens":0,"total_tokens":0,"completion_tokens_details":null,"prompt_tokens_details":null}}
-> http 200 in 0.212458s

$ curl -s http://127.0.0.1:35334/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"guardrails":["loopback-probe"]}'
{"id":"msg_51b81a2e-a458-4bbb-8e67-5c538fd44ad0","type":"message","role":"assistant","content":[{"type":"text","text":"guardrail could not reach loopback: Blocked URL: URL targets a blocked address (127.0.0.1). If this is a legitimate internal service, add the host to `user_url_allowed_hosts` in litellm_settings."}],"model":"gpt-6-luna","stop_reason":"end_turn","usage":{"input_tokens":0,"output_tokens":0}}
-> http 200 in 0.066397s

$ curl -s http://127.0.0.1:35334/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","input":"hi","guardrails":["loopback-probe"]}'
{"id":"resp_4010a589-536f-436b-b266-367d0e7065f2","created_at":1790397606,"error":null,"incomplete_details":null,"instructions":null,"metadata":null,"model":"gpt-6-luna","object":"response","output":[{"content":[{"type":"text","text":"guardrail could not reach loopback: Blocked URL: URL targets a blocked address (127.0.0.1). If this is a legitimate internal service, add the host to `user_url_allowed_hosts` in litellm_settings."}]}],"parallel_tool_calls":null,"temperature":null,"tool_choice":null,"tools":null,"top_p":null,"max_output_tokens":null,"previous_response_id":null,"reasoning":null,"status":"completed","text":null,"truncation":null,"usage":{"input_tokens":0,"input_tokens_details":null,"output_tokens":0,"output_tokens_details":null,"total_tokens":0,"cost":null},"user":null,"store":null}
-> http 200 in 0.032011s

$ # loopback service access log: hits during the three calls above
0 hits

Case 2: sync busy loop with timeout: 5, two concurrent requests (one per worker) and a liveliness probe 3s in

$ for i in 1 2; do curl -s http://127.0.0.1:35334/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["busy-loop"]}' -o busy_$i.json -w "busy $i: http %{http_code} in %{time_total}s\n" & sleep 0.5; done
$ sleep 3; curl -s http://127.0.0.1:35334/health/liveliness -w " <- liveliness: http %{http_code} in %{time_total}s\n"; wait; cat busy_1.json
"I'm alive!" <- liveliness: http 200 in 0.014826s
busy 1: http 500 in 5.066307s
busy 2: http 500 in 5.176780s
{"error":{"message":"Custom code guardrail 'busy-loop' exceeded its 5s execution timeout","type":"internal_server_error","param":null,"code":"500"}}

Case 3: the same busy loop on /v1/messages and /v1/responses

$ curl -s http://127.0.0.1:35334/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"guardrails":["busy-loop"]}'
{"type":"error","error":{"type":"api_error","message":"Custom code guardrail 'busy-loop' exceeded its 5s execution timeout"}}
-> http 500 in 5.114956s

$ curl -s http://127.0.0.1:35334/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","input":"hi","guardrails":["busy-loop"]}'
{"error":{"message":"Custom code guardrail 'busy-loop' exceeded its 5s execution timeout","type":"internal_server_error","param":null,"code":"500"}}
-> http 500 in 5.061354s

Case 4: admin test endpoint with an infinite loop (curl gives up after 20s)

$ curl -s --max-time 20 -X POST http://127.0.0.1:35334/guardrails/test_custom_code -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"custom_code":"def apply_guardrail(inputs, request_data, input_type):\n    while True:\n        pass","test_input":{"texts":["hi"]},"input_type":"request"}'; echo "curl exit=$?"
{"success":false,"result":null,"error":"Execution timeout: code took longer than 5 seconds","error_type":"execution"}
-> http 200 in 5.102163s
curl exit=0

Case 5: async guardrail that loops without ever awaiting, timeout: 5 (curl gives up after 30s)

$ curl -s --max-time 30 http://127.0.0.1:35334/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["async-busy-loop"]}'; echo "curl exit=$?"
{"error":{"message":"Custom code guardrail 'async-busy-loop' exceeded its 5s execution timeout","type":"internal_server_error","param":null,"code":"500"}}
-> http 500 in 5.639149s
curl exit=0

Case 6: is the proxy still alive after cases 4 and 5?

$ curl -s --max-time 5 http://127.0.0.1:35334/health/liveliness -w " <- liveliness: http %{http_code} in %{time_total}s\n"; echo "curl exit=$?"
"I'm alive!" <- liveliness: http 200 in 0.005205s
curl exit=0

Case 7: same guardrail with litellm_settings.user_url_allowed_hosts: ["127.0.0.1:40023"] (second proxy on :46446)

$ curl -s http://127.0.0.1:46446/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-6-luna","messages":[{"role":"user","content":"hi"}],"guardrails":["loopback-probe"]}'
{"id":"chatcmpl-da6b0cfe-5cbf-4da6-a740-dcc337647c94","created":1790397680,"model":"gpt-6-luna","object":"chat.completion","system_fingerprint":null,"choices":[{"finish_reason":"content_filter","index":0,"message":{"content":"guardrail reached loopback: HTTP 200 body=internal-only marker: this loopback service is not reachable from the internet\n","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields":null}}],"usage":{"completion_tokens":0,"prompt_tokens":0,"total_tokens":0,"completion_tokens_details":null,"prompt_tokens_details":null}}
-> http 200 in 0.025744s
$ # loopback service access log: hits during the call above
1 hits
  127.0.0.1 - - [25/Sep/2026 21:41:20] "GET /internal.txt HTTP/1.1" 200 -

Observations from the QA legs and the risk rig that the diff does not show:

• Streaming post_call errors end without [DONE]; pre-existing, leaves alone
• Streaming error chunk carries "type": "None"; pre-existing, leaves alone
• /guardrails/apply_guardrail answers 500 to a block; pre-existing, leaves alone
• 3M-iteration for loop 0.2s before, 0.5s after; PR causes
• Test endpoint now awaits a returned coroutine; PR causes, intended
• Async busy loop still stalls requests until deadline; PR improves

Type

🐛 Bug Fix

Caveats (if any)

Medium

• An async guardrail that busy-loops without awaiting holds the event loop until its deadline (default 30s), then fails its request; before this PR it held the loop forever
• A sync guardrail stuck inside one long C-level call (a catastrophic regex, a huge integer power) cannot be interrupted until that call returns
  • the request still fails at the deadline and the worker thread is logged as stuck; it keeps one core busy until the call ends
  • in an async guardrail the same call runs on the event loop, so it also holds every other request until it returns, as it did before this PR; only once it returns does the deadline fail the request
• An async guardrail that keeps swallowing cancellation is given up after 1s of retries, logged, and left to finish on its own; its result is discarded

Low

• Guardrails that legitimately run longer than 30s need timeout raised in their litellm_params
• Non-GET requests never follow redirects when validation is on; a 3xx is returned as the response
• Module-level guardrail code compiles synchronously on the thread that loads the guardrail, so a busy loop there holds that thread for up to timeout before the load fails; a config-file guardrail like that now stops startup after timeout seconds instead of hanging it
• An allowlisted host still has to resolve through DNS, since the validator resolves before it consults the allowlist; a name only the proxy's own hosts file knows needs user_url_validation: false
• Each sync guardrail run starts one daemon thread, and every loop iteration in guardrail code pays one budget check
• The validator does not yet block the NAT64 range (64:ff9b::/96) or re-resolve a host between validation and the HTTPS connect; tracked as a follow-up

CI reds this PR did not introduce, each matched to main:

• GitHub Actions misc / Run tests (not a required check) fails tests/unit/test_model_prices_schema.py::test_price_map_has_no_duplicate_keys on the duplicate cost-map key openrouter/perceptron/perceptron-mk1.5; main's own runs at 4eb340a, d08746f and c822c7f fail the same way, main removed the duplicate in 4d7aa89 (fix(cost-map): remove duplicate openrouter/perceptron/perceptron-mk1.5 entry #43273) after this branch's merge base, and this PR does not touch the cost map
• CircleCI local_testing_part1 fails tests/local_testing/test_get_model_info.py::test_get_model_info_bedrock_cross_region_capability_parity (supports_prompt_caching differs between us.anthropic.claude-mythos-preview and anthropic.claude-mythos-preview in the branch's cost map) at both tips and on its rerun (job 2222678); main's pipeline 90441 at e0fb89b fails the same test and main passes from pipeline 90478 on, after 4179860 (fix(cost-map): retirement dates, chatgpt reasoning flags, bing pricing, bedrock mantle and mythos, azure gpt-5.6 alias, anthropic batch rates, new nebius, openrouter and xai rows #42951) aligned the two entries; the first run at 5372792 (job 2220583) also failed test_get_model_info_bedrock_region, a state-leaking test main fixed in 7fc2206 (test: fix stale and state-leaking tests red on scheduled CircleCI #43266), and that test passed on the rerun
• CircleCI using_litellm_on_windows times out in its Install Dependencies step before any test runs (jobs 2220539 and 2220581 at this branch's two tips), as it does on main's pipeline 90441 (job 2218756); 7fc2206 (test: fix stale and state-leaking tests red on scheduled CircleCI #43266) raised that step's no-output limit on main after this branch's merge base, and main passes from pipeline 90478 on; its rerun (job 2222677) times out in the same Install Dependencies step at the same 10 minute no-output limit, right after the botocore and basedpyright downloads
• CircleCI local_testing_part2 failed tests/local_testing/test_streaming.py::test_openai_stream_options_call_text_completion once at 649e590 with a pydantic MockValSer serializer error; it passes on main's pipeline 90441 and touches no guardrail code; its re-run at 5372792 (CircleCI job 2220610) passed
• CircleCI unit fails 52 tests at 5372792 (job 2220608), none in a file this PR touches; 38 of them fail identically on main's own unit jobs 2218776 (e0fb89b), 2220817 (99655b6) and 2221978 (115668f), and the other 14 (test_prometheus_labels and test_prometheus_service_tier_label registry state, a test_token_counter event-loop lag check, a test_http_handler finalizer, two leaked-mock test_get_end_user_object cases and one test_complexity_router case) are the state-leak class main fixed in 7fc2206 (test: fix stale and state-leaking tests red on scheduled CircleCI #43266) and 14f4c34 (fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts #43294, tests/unit/conftest.py and those test files) after this branch's merge base; main's pipeline 90510 at d18fcb0, which carries both, fails one unrelated test (test_async_shadow_does_not_inherit_primary_deployment_tags); its rerun (job 2222676) fails 42 tests, 39 of them shared with the first run and matched to main the same way, with 13 of the first run's 14 state-leak cases passing this time; the 3 rerun-only failures (test_token_counter::test_huggingface_count_in_a_worker_thread_leaves_the_event_loop_free, a wall-clock event-loop lag check, and the two test_per_user_oauth_cache::TestMCPServerNewFields needs_user_oauth_token cases, which passed in the first run) sit in files this PR does not touch and pass on main's job 2222544
• CircleCI langfuse_logging_unit_tests hits the 15 minute no-output limit at tests/local_testing/test_alangfuse.py::test_langfuse_logging_without_request_response[False] RETRY (job 2220593); main's jobs 2218773 (e0fb89b), 2220819 (99655b6) and 2221956 (115668f) hang at the same line, and main's pipeline 90510 at d18fcb0 passes after 14f4c34 (fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts #43294) rewrote that test against a local Langfuse; its rerun (job 2222674) hangs at the same RETRY line and hits the same 15 minute limit
• CircleCI proxy_store_model_in_db_tests fails tests/store_model_in_db_tests/test_callbacks_in_db.py::test_e2e_langfuse_callbacks_in_db (the request is not found in Langfuse traces) at both tips and on its rerun (job 2222675); main's pipeline 90441 (job 2218794) fails the same test, and main passes from pipeline 90478 on after e11c3f5 (test(integration): port langfuse callbacks-in-db coverage to the local harness #43282) moved that coverage to the local integration harness
• CircleCI build_and_test fails the two ocr cases of tests/test_rust_python_harness.py::test_implemented_namespace_case_modules_remain_importable (job 2220589), modules refactor(ocr): remove the Python OCR execution path and require the Rust route #43081 removed; main's pipeline 90441 (job 2218790) fails them the same way, and main passes from pipeline 90478 on after 7fc2206 (test: fix stale and state-leaking tests red on scheduled CircleCI #43266) dropped them from the parametrize list; the rerun at 5372792 (job 2222679) fails the same two cases and nothing else
• CircleCI integration-extensions (integration workflow) fails test_passthrough_worker_sigkill_leaves_sibling_serving_and_logging and test_native_gemini_keeps_streaming_after_one_worker_is_killed, both worker-kill chaos tests (job 2220567); main's integration-extensions is red on every pipeline from 90394 to 90510 with a chaos test of that class (job 2218749 fails the same passthrough sigkill test, job 2222504 fails test_burst_survives_one_worker_kill); its rerun (job 2222681) was still running when this section was written, and the job is not a required check

Live PR risk (649e590)

Both sides booted with --num_workers 2, real OpenAI calls (gpt-6-luna), a loopback marker service on :40023 and a forwarding stub in front of a generic_guardrail_api guardrail; base is the merge base 4adbc13, head is 649e590, and the same scenarios re-run on 8017f9469c (head merged into main at 4eb340a) answered identically to the head leg. 5372792 adds tests only, so the code under test is 649e590 byte for byte. S-numbers refer to the risk rig scenarios, cases to the QA legs above

Breaking

None observed. Every path that worked on base answers the same on head: a call with no guardrail (200 both), a generic_guardrail_api guardrail whose vendor answers a 302 (200 both, the vendor stub logged the same 2 hits on both sides, S6), real https destinations through http_get and http_post (the OpenAI 401 and the httpbin echo came back identical, S7), a post_call guardrail raising a plain exception on a non-streaming and a streaming call (the 500 body and the streaming error chunk are identical on both sides, S9a), and a sync guardrail returning an un-awaited coroutine at request time (200 with the model's answer on both sides, S9b)

Backward incompatible

• A guardrail run longer than timeout (default 30s) now fails its request: 500 exceeded its 5s execution timeout on pre_call and during_call (S1, S5, cases 2 and 3), 500 on a non-streaming post_call (S2), and an error chunk that closes the stream after the 200 on a streaming post_call (S3); on base the same requests waited 21s to 57s for the loop to end. The configurable timeout is what the ticket behind this PR asks for; Decision 2 sets fail closed
• http_request, http_get and http_post to a private, loopback or link-local address now return {"success": false, "error": "Blocked URL: ..."} unless the host is in user_url_allowed_hosts (case 1: 0 loopback hits on head against 3 on base; case 7 shows the allowlist letting the host through). The block is what the ticket behind this PR asks for
• A caller-supplied Host header is dropped from http_* requests while validation is on, and passes through untouched with validation off (test_caller_host_header_never_reaches_the_validated_destination, test_caller_headers_pass_through_untouched_when_url_validation_is_disabled). Decision 9; ships with this PR's review
• POST /guardrails/test_custom_code now awaits a coroutine a sync apply_guardrail returns and reports its result: head answers with the primitive's blocked-URL result, base answered {"action": "allow", "warning": "Expected dict result, got coroutine. Treating as allow."} (S8). The request-time path already awaited it on both sides (S9b), so the endpoint now matches it
• Every loop iteration in guardrail code pays a budget check: the 3M-iteration for loop bench went from 0.20s to 0.50s per run (S4). Decision 6

Regression risk

• A sync guardrail stuck in one long C-level call keeps its worker thread busy past the deadline (Medium caveat); covered by the unit tests only (_spin_swallowing_exceptions, test_custom_code_bounded_execution.py), not driven on the rig, since the rig has no primitive that blocks in C for seconds
• The POST /guardrails compile-failure rollback is covered by test_guardrail_endpoints.py and was not driven on a DB-backed proxy

Dependency graph

Dependent	Entrypoint	State
CustomCodeGuardrail.apply_guardrail pre_call, during_call, post_call	/v1/chat/completions, /v1/messages, /v1/responses	verified live (cases 1 to 3, S2, S3, S5)
POST /guardrails/apply_guardrail	admin route	verified live (S1)
POST /guardrails/test_custom_code	admin route	verified live (case 4, S8)
POST /guardrails compile-failure rollback	admin route	tested, not driven live
async_safe_get and validate_url, shared with other callers	generic_guardrail_api vendor call	verified live, unchanged (S6)
GuardrailCallback httpx client params	every http_* primitive	verified live (S7)
streaming post_call error path	/v1/chat/completions with stream: true	verified live (S3, S9a)
LitellmParams.timeout reuse	config load, Admin UI guardrail form	tested (test_init_guardrails.py), UI not driven
/health/liveliness during a busy loop	health route	verified live (case 2)

Not verified

• POST /guardrails answering 400 for a non-custom_code guardrail whose parameters fail validation (no DB-backed proxy booted)
• The Admin UI guardrail form showing timeout for a custom_code guardrail
• MCP, realtime and logging_only guardrail modes running custom code
• Budget-check cost beyond the one 3M-iteration bench

Decisions

1. Default execution timeout 30s, matching the default http_request timeout so one default HTTP call fits; alternatives were 5s (the admin test endpoint's budget) and 60s (the http_request cap)
2. Timeout fails closed: the request errors (CustomCodeExecutionError, HTTP 500 pre-call) instead of being allowed through
3. timeout reuses LitellmParams.timeout (already shown in the Admin UI) instead of a new execution_timeout key; zero, negative and NaN values are rejected at load
4. GET follows redirects with per-hop validation via async_safe_get; POST/PUT/PATCH/DELETE never follow redirects, since a validated destination that redirects elsewhere would re-open the hole
5. Sync guardrail code moved off the event loop onto a daemon worker thread with PyThreadState_SetAsyncExc interruption, instead of a subprocess or a second event loop (the cached aiohttp-backed httpx client is loop-bound)
6. Budget checks are compiled into loop headers (while tests, for and comprehension iterators) rather than into every statement, so straight-line code pays nothing and only loops are checked
7. A result that arrives after the deadline counts as a timeout, so code that catches the interrupt cannot turn it into an allow
8. POST /guardrails catches a compile failure by name and rolls back with a 400, while a config-file guardrail that does not compile still stops startup as before; only a bad parameter such as timeout: 0 is a ValueError, which the startup loader has skipped with a warning since feat(guardrails): add Lakera v2 skip-message honoring and advisory (inject_system_message) mode #34940
9. A caller-supplied Host header is stripped from http_* requests while validation is on, so the validated host is the only one on the wire; with validation off the headers pass through untouched, as before
10. GET keeps the shared client's 5s connect timeout and only the total timeout is per call, matching the previous behavior; the other methods pass the per-call timeout through as they did before

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[High risk] Adds execution timeout and SSRF validation to custom guardrail code.

The PR appears safe to merge based on this review; the latest changes introduce no established blocking issue.

Findings

1.  Long regex calls outlive timeouts ▶
2.  Timed-out tasks keep running ▶

Summary

The PR adds destination validation and execution limits for custom-code guardrails, improves guardrail creation and testing errors, and adds coverage for those behaviors. Since the previous review, the changes are tests only; no new actionable issue was found.

_{Reviews (3) · Last reviewed commit: "test(guardrails): cover the http_request..."}

[greptile-apps]

Long regex calls outlive timeouts

If custom code calls the exposed regex_match primitive with a pattern that takes a long time to evaluate, these loop checks and thread interrupts cannot stop the regex while it is running. A worker can keep consuming CPU after the guardrail reports a timeout. In an async guardrail, the same call can stall the event loop before its timeout runs, so the new limit does not bound this supported primitive.

[mateo-berri]

Named as a Medium caveat: Python cannot interrupt a C-level call. The request still fails at the deadline; before, it stalled the proxy forever.

[greptile-apps]

The caveat correctly covers synchronous guardrails: their worker thread keeps the event loop responsive, although the C call may continue consuming CPU.

However, async guardrails take a different path. inspect.iscoroutinefunction() runs them as an event-loop task, while regex_match performs synchronous re.search(). A catastrophic regex executed before the next await can therefore block both the timeout waiter and other requests until the regex returns.

The caveat should also mention this async case, or synchronous primitives such as the regex helpers should be moved off the event loop.

[greptile-apps]

Timed-out tasks keep running

If an async guardrail catches cancellation and continues awaiting work, _cancel stops trying after one second and this code reports a timeout while the task is still alive. Repeated requests can accumulate tasks that retain request data and resources. The added test confirms that a task survives this path.

[mateo-berri]

Named as a Medium caveat: after 1s of refused cancellation the task is logged and abandoned, its result discarded. Before, nothing bounded it at all.

[mateo-berri]

Intentional: sandboxed code can raise SystemExit or a bare BaseException, and this converts it to SandboxExit so it cannot stop the event loop.

[codecov]

Codecov Report

❌ Patch coverage is 98.34711% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...s/guardrail_hooks/custom_code/bounded_execution.py	97.29%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_custom_code_guardrail_ssrf_and_timeout (5372792) with main (d08746f)¹}

Footnotes

1. No successful run was found on main (e11c3f5) during the generation of this report, so d08746f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5372792. Configure here.}

[mateo-berri]

LGTM