Revert "fix: langfuse trace leak key on model params"#23868
Conversation
yuneng-jiang
merged commit b4c9c8a
into
litellm_internal_dev_03_16_2026
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR reverts #22188 ("fix: langfuse trace leak key on model params"), which had introduced whitelist-based sanitisation of Key changes and concerns:
Recommendation: Rather than a full revert, identify the specific tests that were failing and fix the filtering logic to accommodate them while keeping the security protections in place. The whitelist approach in Confidence Score: 1/5
|
| Filename | Overview |
|---|---|
| litellm/integrations/langfuse/langfuse.py | Reverts whitelist-based model-parameter sanitisation; raw optional_params (including api_key, authorization, headers) are now forwarded to Langfuse in both v1 and v2 logging paths, reintroducing the credential-leak vulnerability that PR #22188 fixed. |
| litellm/litellm_core_utils/litellm_logging.py | scrub_sensitive_keys_in_metadata now omits removal of user_api_key_auth, callback_settings, and logging scrubbing for user_api_key_auth_metadata / user_api_key_team_metadata, reintroducing credential exposure in all logging integrations. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[LiteLLM request completes] --> B[Logging handler collects kwargs]
B --> C[optional_params extracted]
C --> D{This PR: pop secret_fields only}
D --> E[optional_params still contains:\napi_key, api_base,\nauthorization, headers]
E --> F[Passed as modelParameters\nto Langfuse v1 & v2]
F --> G[🚨 Credentials stored\nin Langfuse trace]
B --> H[scrub_sensitive_keys_in_metadata]
H --> I{user_api_key_metadata?}
I -- yes --> J[Scrub 'logging' key only\nin user_api_key_metadata]
I -- no --> K[No-op]
J --> L[user_api_key_auth NOT removed\ncallback_settings NOT removed\nother metadata fields NOT scrubbed]
L --> G
style G fill:#f55,color:#fff
style D fill:#ffa500,color:#000
style L fill:#ffa500,color:#000
Last reviewed commit: 467706e
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| model=model_name, | ||
| modelParameters=sanitized_model_params, | ||
| modelParameters=optional_params, | ||
| prompt=input, |
There was a problem hiding this comment.
After this revert, the raw optional_params dict (with only secret_fields popped) is passed directly to Langfuse as modelParameters. Fields like api_key, api_base, authorization, and headers can still leak to Langfuse traces.
The existing test test_langfuse_model_parameters_no_secret_leakage explicitly verifies that these fields must NOT appear in the logged parameters:
# Sensitive params that must NOT leak
"api_key": "sk-secret-key-12345",
"api_base": "https://my-private-endpoint.com",
"authorization": "Bearer sk-another-secret",
"headers": {"X-Api-Key": "secret-header-value"},Popping only secret_fields is insufficient — these other keys will still pass through to Langfuse. The whitelist approach from ModelParamHelper.get_standard_logging_model_parameters() was what prevented leakage of all non-standard keys.
| "model": model_name, | ||
| "model_parameters": sanitized_model_params, | ||
| "model_parameters": optional_params, | ||
| "input": input if not mask_input else "redacted-by-litellm", |
There was a problem hiding this comment.
Same credential leak issue as the v1 path: optional_params is passed directly to Langfuse's generation() without whitelist filtering. Fields such as api_key, authorization, and headers present in optional_params will be recorded in Langfuse traces.
The removed logic previously fell back to ModelParamHelper.get_standard_logging_model_parameters() when no standard_logging_object was available, providing a last-resort sanitisation layer. That safety net no longer exists.
| ## check user_api_key_metadata for sensitive logging keys | ||
| cleaned_user_api_key_metadata = {} | ||
| if "user_api_key_metadata" in metadata and isinstance( | ||
| metadata["user_api_key_metadata"], dict | ||
| ): | ||
| if metadata_field in metadata and isinstance(metadata[metadata_field], dict): | ||
| for sensitive_key in _sensitive_keys: | ||
| metadata[metadata_field].pop(sensitive_key, None) | ||
|
|
||
| ## remove user_api_key_auth entirely - contains full auth object with nested credentials | ||
| metadata.pop("user_api_key_auth", None) | ||
| for k, v in metadata["user_api_key_metadata"].items(): | ||
| if k == "logging": # prevent logging user logging keys | ||
| cleaned_user_api_key_metadata[k] = ( | ||
| "scrubbed_by_litellm_for_sensitive_keys" | ||
| ) | ||
| else: | ||
| cleaned_user_api_key_metadata[k] = v | ||
|
|
||
| litellm_params["metadata"] = metadata | ||
| metadata["user_api_key_metadata"] = cleaned_user_api_key_metadata | ||
| litellm_params["metadata"] = metadata |
There was a problem hiding this comment.
The original scrub_sensitive_keys_in_metadata removed three categories of sensitive data that are no longer handled after this revert:
-
user_api_key_authentire key — the old code explicitly calledmetadata.pop("user_api_key_auth", None)with the comment "contains full auth object with nested credentials". That object (aUserAPIKeyAuthinstance) may include team secrets, org tokens, and other nested credentials. It is no longer removed. -
callback_settings— removed from all three metadata fields (user_api_key_metadata,user_api_key_auth_metadata,user_api_key_team_metadata) previously. These can contain callback API keys (e.g., Langfuse secret keys). Onlyuser_api_key_metadatais touched now, and only theloggingkey, notcallback_settings. -
user_api_key_auth_metadataanduser_api_key_team_metadata— theloggingkey inside these two fields is no longer scrubbed. This means per-team / per-key logging credentials could reach Langfuse.
Additionally, the litellm_params["metadata"] = metadata write-back is now inside the if "user_api_key_metadata" in metadata block. If the incoming metadata dict was resolved via the or {} fallback (i.e., it's a freshly allocated dict), any mutations will not be persisted back to litellm_params when user_api_key_metadata is absent.
1 participant
Reverts #22188
https://app.circleci.com/pipelines/github/BerriAI/litellm/69346/workflows/b702f2b3-b3a7-4bf3-b6ca-b180b07b5a1c/jobs/1414382/tests
Please take a look at the failing tests when you get a chance