fix(logging): treat an empty prompt_id as no prompt in manager dispatch

[bharadwaj-pendyala]

TLDR

Problem this solves:

• An empty prompt_id silently drops cache_control_injection_points
• Two gates disagree on whether "" names a prompt

How it solves it:

• Dispatch reads a blank prompt_id the way the entry gate does
• A manager that declines no longer shadows the cache-control hook

User Flow

Before: a developer whose client sends an empty prompt id gets no prompt caching, and nothing tells them why

1. The admin sets callbacks: ["dotprompt"] and gives claude-opus-4-5 a cache_control_injection_points: [{role: system, location: message}]
2. The developer sends POST http://localhost:4000/v1/chat/completions with a system message, a user message, and "prompt_id": ""
3. The response is HTTP 200 with a normal answer, so nothing looks broken
4. The provider never receives the system block marked for caching, so http://localhost:4000/ui/?page=logs bills the whole prompt as a write on every call and never as a cache read
5. The developer removes the prompt_id field and resends. That request does get the cache marker, so the empty string is the only difference

After: an empty prompt id behaves exactly like an absent one, and the prefix caches

1. The admin keeps the same config
2. The developer sends the same POST with "prompt_id": ""
3. The response is HTTP 200 with the same answer
4. The provider now receives the system block carrying "cache_control": {"type": "ephemeral"}, so http://localhost:4000/ui/?page=logs bills the next identical call as a cache read

Relevant issues

Follow-up to #37469

That report is the prompt-less case, and 4829bb3 already fixed it on litellm_internal_staging. I re-verified: on 35416c702d a call with no prompt_id gets the cache marker. What survives is the empty-string case, which is what this PR narrows to, so it does not close the issue on its own

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

should_run_prompt_management_hooks decides whether to run the prompt hooks at all, and it gates on if prompt_id, so "" reaches dispatch as a call that names no prompt. get_custom_logger_for_prompt_management then gated on prompt_id is None, so "" skipped the guard, dotprompt was selected, and it returned the messages untouched. The cache-control hook sits after that in the same chain and never ran

This is local preprocessing that happens before any provider request is built, so the run below drives real HTTP through the full SDK against an upstream that records what it receives. I have no provider credentials on this machine, so the upstream is a stub on 127.0.0.1 rather than Anthropic. What it proves is the system block on the wire, which is exactly the bytes the provider would have been sent

Shared setup, both cases:

mkdir -p /tmp/prompts
printf -- '---\nmodel: claude-opus-4-5\n---\nyou are a stem tutor\n' > /tmp/prompts/stem.prompt
export DOTPROMPT_DIRECTORY=/tmp/prompts

cat > /tmp/upstream.py <<'PY'
import json
from http.server import BaseHTTPRequestHandler, HTTPServer

CANNED = {"id": "msg_stub", "type": "message", "role": "assistant", "model": "claude-opus-4-5",
          "content": [{"type": "text", "text": "Hi from the stub upstream."}],
          "stop_reason": "end_turn", "usage": {"input_tokens": 12, "output_tokens": 6}}

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        open("/tmp/last_request.json", "wb").write(self.rfile.read(int(self.headers["content-length"])))
        payload = json.dumps(CANNED).encode()
        self.send_response(200)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(payload)))
        self.end_headers()
        self.wfile.write(payload)
    def log_message(self, *a): pass

HTTPServer(("127.0.0.1", 8899), Handler).serve_forever()
PY

cat > /tmp/call.py <<'PY'
import json, sys, litellm
litellm.callbacks = ["dotprompt"]
prompt_id = sys.argv[1] if len(sys.argv) > 1 else None
kwargs = {} if prompt_id is None else {"prompt_id": prompt_id}
r = litellm.completion(
    model="anthropic/claude-opus-4-5", api_key="sk-ant-stub", api_base="http://127.0.0.1:8899",
    messages=[{"role": "system", "content": "You are helpful."},
              {"role": "user", "content": "Hello"}],
    cache_control_injection_points=[{"role": "system", "location": "message"}], **kwargs)
print("HTTP 200:", r.choices[0].message.content)
print("system sent upstream:", json.dumps(json.load(open("/tmp/last_request.json"))["system"]))
PY

python /tmp/upstream.py &

Before (35416c7)

no prompt_id, the case #37469 reported

1. python /tmp/call.py
2. Output, already correct here because of 4829bb3:

HTTP 200: Hi from the stub upstream.
system sent upstream: [{"type": "text", "text": "You are helpful.", "cache_control": {"type": "ephemeral"}}]

empty prompt_id

1. python /tmp/call.py ""
2. Output, note the missing cache_control:

HTTP 200: Hi from the stub upstream.
system sent upstream: [{"type": "text", "text": "You are helpful."}]

After (1514baa)

no prompt_id, the case #37469 reported

1. python /tmp/call.py
2. Output, unchanged:

HTTP 200: Hi from the stub upstream.
system sent upstream: [{"type": "text", "text": "You are helpful.", "cache_control": {"type": "ephemeral"}}]

empty prompt_id

1. python /tmp/call.py ""
2. Output:

HTTP 200: Hi from the stub upstream.
system sent upstream: [{"type": "text", "text": "You are helpful.", "cache_control": {"type": "ephemeral"}}]

Type

🐛 Bug Fix

Caveats (if any)

• GenericPromptManager accepts ""; it stops being selected for it
• Whitespace-only ids still count as a real prompt
• Behaviour for a populated prompt_id is unchanged

On the first one, worth a maintainer's call before merge. GenericPromptManager.should_run_prompt_management returns True for "" because it tests prompt_id is not None, so today it is selected and then compiles against an empty id. After this change it is skipped, same as every other in-tree manager. I read that as the predicate being wrong rather than behaviour to keep, but if you want "" to stay a valid id for that manager, the fix belongs in the entry gate at should_run_prompt_management_hooks instead and I will move it

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

The PR aligns empty-string prompt_id handling with the existing prompt-less dispatch behavior.

• Treats an empty prompt ID as absent when selecting registered prompt managers.
• Adds regression coverage showing cache-control injection still runs for an empty prompt ID.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported Arize prompt-less dispatch failure is prevented because the new falsy guard consults the manager’s no-prompt predicate and skips Arize before compilation.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/litellm_logging.py	Updates prompt-manager fallback selection so empty prompt IDs follow the same path as absent IDs.
tests/test_litellm/litellm_core_utils/test_litellm_logging.py	Adds focused coverage confirming cache-control metadata is injected when prompt_id is empty.

_{Reviews (2): Last reviewed commit: "fix(logging): treat an empty prompt_id a..." | Re-trigger Greptile}

[greptile-apps]

Comments duplicate dispatch logic

These comments and related test docstrings restate behavior already expressed by the code, adding maintenance overhead and avoidable documentation drift

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing bharadwaj-pendyala:fix/prompt-manager-claims-promptless-calls (bcc5c90) with litellm_internal_staging (ff4b558)}

[bharadwaj-pendyala]

Greptile's blocking point is real, but it predates this PR and the dispatcher is not where it can be fixed.

ArizePhoenixPromptManager overrides the ownership predicate to claim everything, and says so on purpose:

# litellm/integrations/arize/arize_phoenix_prompt_manager.py:353-365
def should_run_prompt_management(self, prompt_id, prompt_spec, dynamic_callback_params) -> bool:
    """
    Determine if prompt management should run based on the prompt_id.

    For Arize Phoenix, we always return True and handle the prompt loading
    in the _compile_prompt_helper method.
    """
    return True

So the probe this PR adds asks Phoenix whether it owns a prompt-less cache-control call, and Phoenix volunteers. PromptManagementBase.get_chat_completion_prompt then raises one line before it would have consulted that predicate again:

# litellm/integrations/prompt_management_base.py:167-169
if prompt_id is None:
    raise ValueError("prompt_id is required for Prompt Management Base class")
if not self.should_run_prompt_management(

I checked whether this PR put that path there. Register the manager, send cache_control_injection_points with no prompt_id, print what the dispatcher returns and what happens when you use it.

On 54754e4, this branch:

selected: ArizePhoenixPromptManager
get_chat_completion_prompt raised: ValueError prompt_id is required for Prompt Management Base class

On c696fdfb05, the merge base:

selected: ArizePhoenixPromptManager
get_chat_completion_prompt raised: ValueError prompt_id is required for Prompt Management Base class

Identical. Before this change the unconditional prompt_management_loggers[0] fallback handed the call to Phoenix. After it, the probe hands the call to Phoenix because Phoenix asks for it. Same manager, same raise.

The base-commit run really did use base code, not the editable install:

$ PYTHONPATH=/tmp/litellm-base .venv/bin/python -c "
import litellm, inspect
from litellm.litellm_core_utils.litellm_logging import Logging
src = inspect.getsource(Logging.get_custom_logger_for_prompt_management)
print('litellm from:', litellm.__file__)
print('has_old_fallback:', 'fallback' in src)
print('has_new_comment:', 'must not block the hook' in src)"
litellm from: /private/tmp/litellm-base/litellm/__init__.py
has_old_fallback: True
has_new_comment: False

Phoenix is the only prompt manager in the tree that answers this way. Every other one already declines a prompt-less call, which is why the probe fixes them:

Manager	Predicate on prompt_id=None
Dotprompt	if prompt_id is None: return False (dotprompt_manager.py:96)
GitLab	return prompt_id is not None (gitlab_prompt_manager.py:472)
BitBucket	return prompt_id is not None (bitbucket_prompt_manager.py:422)
Generic	False unless prompt_spec carries provider params (generic_prompt_manager.py:218)
Langfuse	if prompt_id is None: return False (langfuse_prompt_management.py:218)
Arize Phoenix	return True

Those five stop being selected, so AnthropicCacheControlHook gets the request, which is #37469.

The Phoenix fix is one line in its predicate, but it reverses a documented decision inside an integration this PR does not otherwise touch, and I would rather not bury that in a dispatcher change. Happy to send it as a follow-up, or to add it here if you would rather review both at once. Tell me which you prefer and I will do that.

[bharadwaj-pendyala]

Rescoped: 4829bb3 landed the prompt-less fix, so this now covers only the empty-string prompt_id it left. Rebased on staging

[bharadwaj-pendyala]

@greptileai

[bharadwaj-pendyala]

Rebased onto litellm_internal_staging at f005afa. The branch was 239 commits behind and GitHub had it as DIRTY, so it could not have merged without this.

The conflict was in the test file only, and it was additive on both sides: staging appended three newrelic OTel v2 tests, this branch appended one. Both kept, nothing dropped. The one-line source change survived untouched:

-            if prompt_id is None and not self._prompt_manager_runs_without_prompt_id(
+            if not prompt_id and not self._prompt_manager_runs_without_prompt_id(

Re-proved on the new base rather than assuming the rebase was safe. Reverting that single line turns the new test red, restoring it turns it green, and the whole file passes:

$ pytest tests/test_litellm/litellm_core_utils/test_litellm_logging.py -q
172 passed in 1069.11s

One thing I want to put on the record rather than leave for a reviewer to find, because the diff is small but it moves more than the empty string.

should_run_prompt_management on the Arize Phoenix, GitLab, BitBucket and Generic managers is return prompt_id is not None, so each of them claims "" while declining None. _prompt_manager_runs_without_prompt_id asks with None, so after this change those four stop being selected for prompt_id="":

arize  should_run(None) = False
arize  should_run('')   = True
gitlab should_run(None) = False
gitlab should_run('')   = True

That skip is the point, not a side effect. Arize's own docstring says it "needs a prompt_id to compile, so it declines requests without one", and today it gets handed a call naming no prompt and goes looking for a prompt called "". GitLab is the same shape: decode_prompt_id("") returns "" without raising, so it reaches the fetch and fails there instead of declining up front.

The line above already agrees with that reading. Auto-detection at litellm_logging.py:1019 is gated on if prompt_id and dynamic_callback_params is not None, plain truthiness, so an empty ID is already treated as absent one branch earlier. The fallback loop was the only place still testing is None. This makes the two consistent.

Same reasoning covers the non-str falsy values. prompt_id is typed str | None, but main.py:515 forwards kwargs.get("prompt_id") unvalidated, so 0 and [] can reach the loop. They take the new path too, which matches what the truthiness gate above them already does.

Happy to narrow it to prompt_id != "" if you would rather the change stopped at the reported case.

[bharadwaj-pendyala]

code-quality went red on the rebase and it is not coming from this branch.

The failing step is check_workflow_startup_safety.py, and it objects to a file this PR does not touch:

.github/workflows/test-unit.yml: job `unit` gives pytest 20m but caps the job at 55m.
Setup can use up to 35m plus 5m of runner overhead, so the job deadline would preempt
pytest; raise job-timeout-minutes to at least 60.

The branch changes two files, litellm/litellm_core_utils/litellm_logging.py and its test. Nothing under .github/.

Checked out litellm_internal_staging at f005afa146 on its own, with none of these commits applied, and ran the same script:

$ python ./tests/code_coverage_tests/check_workflow_startup_safety.py
ERROR: Workflow startup invariants violated:
  - .github/workflows/test-unit.yml: job `unit` gives pytest 20m but caps the job at 55m. ...

Same three lines. d1f3778849 (#37804) raised the shard parallelism yesterday and left the unit job at 55 minutes, so every PR rebased onto staging since then inherits this.

Leaving it alone, since bumping job-timeout-minutes in a prompt-dispatch bug fix is not my call to make. Happy to send it as a one-line PR of its own if that helps.

[bharadwaj-pendyala]

The code-quality failure I reported on 23 Aug is gone. It was the stale job-timeout-minutes: 55 entries in .github/workflows/test-unit.yml, which litellm_internal_staging has since raised to 60; my branch was still carrying the old values from the 23 Aug rebase. Rebased onto 891b23d and code-quality now passes (run 32800055891).

One check is still red, and it is not from this PR:

FAILED tests/mcp_tests/test_mcp_logging.py::test_mcp_cost_tracking_per_tool
assert 1.35e-05 == 5.0

The same assertion, same numbers, fails on three unrelated branches from tonight:

• litellm_team_member_budget_stuck_counter (run 32800935799)
• fix-a2a-agent-provider (run 32799555677)
• litellm_dashscope_kimi_minimax_dsv4pro0813 (run 32797110439)

This PR touches Logging.get_custom_logger_for_prompt_management_hooks and nothing in the MCP cost path, so I have left it alone rather than paper over it.

Everything else is green on be0b570. The regression test still brackets the fix on the new base: test_prompt_hooks_skip_prompt_managers_when_prompt_id_is_empty fails with prompt_id is None restored and passes with not prompt_id.