fix(langfuse): log custom_tool_call items in Responses API output

[CNSeniorious000]

TLDR

Problem this solves:

• Langfuse drops Responses API custom tool calls
• Custom tool input needs the existing message-redaction protection

How it solves it:

• Records custom tool identifiers and raw input
• Redacts custom input in both response representations
• Preserves staging's safe parsing of function arguments

User Flow

Before: a developer reviewing a freeform tool call cannot see its payload in Langfuse

1. Send POST http://localhost:47431/v1/responses with an exec custom tool and tool_choice: "required"
2. Receive HTTP 200 with a custom_tool_call containing JavaScript
3. Open the corresponding Langfuse generation: the custom call is missing, even when an accompanying text message appears

After: the same call appears in Langfuse and respects message redaction

1. Send POST http://localhost:47432/v1/responses with the same tool and payload
2. Receive HTTP 200 with a custom_tool_call containing JavaScript
3. Open the corresponding Langfuse generation: the custom call is present; with x-litellm-enable-message-redaction: true, its input is redacted-by-litellm

Relevant issues

Fixes #37430

Affected release

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 on the current tip

Validation at 8926f645f5: both affected test files pass, 86 passed. The four new cases fail against merge base 252c71c0b2. Changed executable lines have 100% coverage. Ruff, type-discipline and test-quality budgets pass without budget changes; the current CI lint passes too

LITELLM_LOCAL_MODEL_COST_MAP=True PYTHONPATH=. python -m pytest \
  tests/test_litellm/integrations/test_langfuse_otel.py \
  tests/test_litellm/litellm_core_utils/test_redact_messages.py -q
# 86 passed

Screenshots / Proof of Fix

Captured 2026-09-16 against two isolated proxies, the merge base on port 47431 and the current tip on 47432. Both used the same live gpt-6-astra deployment and self-hosted Langfuse, with actual provider calls and no mocks. The same request on each proxy returned HTTP 200. The observations below belong to these test proxies, identified separately from the upstream gateway's traces

Shared config, with credentials supplied through environment variables:

model_list:
  - model_name: gpt-6-astra
    litellm_params:
      model: openai/gpt-6-astra
      api_base: http://127.0.0.1:4000/v1
      api_key: os.environ/UPSTREAM_API_KEY
    model_info:
      mode: responses
litellm_settings:
  callbacks: [langfuse_otel]
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST and LANGFUSE_OTEL_HOST for the test project. Save this as custom.json:

{"model":"gpt-6-astra","input":"Use the exec tool to print 6*7 in JavaScript.","tools":[{"type":"custom","name":"exec","description":"Run JavaScript and return stdout"}],"tool_choice":"required","store":false,"reasoning":{"effort":"low"},"max_output_tokens":1024}

For function.json, replace tools with:

[{"type":"function","name":"exec","description":"Run JavaScript","parameters":{"type":"object","properties":{"code":{"type":"string"}},"required":["code"],"additionalProperties":false},"strict":true}]

For text.json, remove tools and tool_choice, and set input to Reply with exactly: 42

Before (252c71c)

Custom tool output

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47431/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' --data-binary @custom.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","custom_tool_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/a57c0dc37adb3d3f" | jq '{id, output}'

   {"id":"a57c0dc37adb3d3f","output":[{"role":"assistant","content":"I’ll print `6 * 7` using JavaScript."}]}

Custom tool with redaction

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47431/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -H 'x-litellm-enable-message-redaction: true' --data-binary @custom.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","custom_tool_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/1b1af6edb86ebe19" | jq '{id, output}'

   {"id":"1b1af6edb86ebe19","output":[{"role":"assistant","content":"redacted-by-litellm"}]}

Function tool with redaction

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47431/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -H 'x-litellm-enable-message-redaction: true' --data-binary @function.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","function_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/82a77dafb63966fb" | jq '{id, output}'

   {"id":"82a77dafb63966fb","output":[{"role":"assistant","content":"redacted-by-litellm"},{"id":"fc_0e0d2eb388296538016aaa107e4b4487d091d700512679d1fe","name":"exec","call_id":"call_3KBeE9ARgEAlGLh6cHfgFge4","type":"function_call","arguments":{}}]}

Plain text control

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47431/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' --data-binary @text.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/5e79a56489ba83b5" | jq '{id, output}'

   {"id":"5e79a56489ba83b5","output":[{"role":"assistant","content":"42"}]}

After (8926f64)

Custom tool output

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47432/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' --data-binary @custom.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","custom_tool_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/ed115755387c8d3e" | jq '{id, output}'

   {"id":"ed115755387c8d3e","output":[{"role":"assistant","content":"I’ll print `6 * 7` using JavaScript."},{"id":"ctc_0ce2f9b6c75257ef016aaa107d8c6487d09fa29987888eb375","name":"exec","call_id":"call_ASagu9uv36LxaL7eGBiGIJBH","type":"custom_tool_call","input":"console.log(6 * 7);\n"}]}

Custom tool with redaction

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47432/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -H 'x-litellm-enable-message-redaction: true' --data-binary @custom.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","custom_tool_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/3ff3f0b2c3e3af99" | jq '{id, output}'

   {"id":"3ff3f0b2c3e3af99","output":[{"role":"assistant","content":"redacted-by-litellm"},{"id":"ctc_09d3b2412d2168e6016aaa107dd4e887d0ab33b52e9bcc9197","name":"exec","call_id":"call_i4IBJFyHMCJYtLPnQ9CvaEzC","type":"custom_tool_call","input":"redacted-by-litellm"}]}

Function tool with redaction

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47432/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -H 'x-litellm-enable-message-redaction: true' --data-binary @function.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message","function_call"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/8fd048d8e23cae6c" | jq '{id, output}'

   {"id":"8fd048d8e23cae6c","output":[{"role":"assistant","content":"redacted-by-litellm"},{"id":"fc_0166aa94101ea74d016aaa107e643087d0a8172ab47e56255f","name":"exec","call_id":"call_j7pPUZBOLEVI6hkiWLjTc1Ft","type":"function_call","arguments":{}}]}

Plain text control

1. Send the request and inspect the returned item types

   curl -sS http://localhost:47432/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' --data-binary @text.json -o response.json -w 'http=%{http_code}\n'
   jq '[.output[].type]' response.json
   # http=200
   # ["message"]

2. Read the test proxy's generation from Langfuse

   curl -sS -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" "$LANGFUSE_HOST/api/public/observations/8efb4e3ca9b850f1" | jq '{id, output}'

   {"id":"8efb4e3ca9b850f1","output":[{"role":"assistant","content":"42"}]}

Type

Bug Fix

Caveats (if any)

Low

• Vertex CI has four identical failures on the target branch
• The auth prefetch test hits an existing cache-expiry flake tracked in test(auth): freeze prefetch cache clock so the org getter cannot miss on a slow runner #41278
• Re-running the auth job requires repository admin rights; the rerun API returned HTTP 403

At 8926f645f5, 82 checks succeeded, two failed and one was skipped. Lint, integrations and core-utils passed. Greptile scored the current tip 5/5, Cursor Bugbot and Veria succeeded, and both review threads are resolved

The target branch's Vertex job and this PR's Vertex job fail the same four embedding-pricing assertions, with the same observed and expected values. They are outside the four files changed here

The proxy-behavior job fails test_join_binds_the_membership_to_the_requested_team with TypeError: object MagicMock can't be used in 'await' expression after the five-second prefetch cache expires. The affected test and implementation are unchanged by this PR; #41278 fixes the test clock on the same target branch

Final Attestation

• The tests cover raw custom output, redacted custom output, both redaction representations, and the existing function-call behavior

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[greptile-apps]

Greptile Summary

The PR now records Responses API custom_tool_call items in Langfuse while redacting their freeform input when message logging is disabled.

• Maps custom tool calls into Langfuse observation output.
• Extends object and dictionary response redaction paths.
• Adds regression coverage for logged and redacted custom tool calls.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/integrations/langfuse/langfuse_otel.py	Adds custom tool-call output mapping while preserving safe JSON parsing for function-call arguments.
litellm/litellm_core_utils/redact_messages.py	Redacts custom tool-call input in both object and dictionary Responses API representations.
tests/test_litellm/integrations/test_langfuse_otel.py	Verifies Langfuse output for custom tool calls with redaction both enabled and disabled.
tests/test_litellm/litellm_core_utils/test_redact_messages.py	Covers custom tool-call input redaction for dictionary and object output forms.

_{Reviews (3): Last reviewed commit: "fix(langfuse): avoid Final assignment in..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing CNSeniorious000:fix/langfuse-otel-custom-tool-call-output (36a226c) with main (c877e0f)¹}

Footnotes

1. No successful run was found on main (a9b9700) during the generation of this report, so c877e0f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[CNSeniorious000]

Pushed two commits and refreshed the PR description:

• 8283be8 — the custom_tool_call branch built its own dict literal, which pushed LIT002 one over the ceiling in the type-discipline gate (langfuse_otel.py:212). Folded it into the neighbouring function_call branch instead of suppressing the rule: the two differ only in which key carries the payload (arguments vs input), so one literal now serves both and the file is back at its base total (21).
• ae54fca — ruff format on the folded branch.

lint is green now. The one red check is misc / Run tests, failing on tests/test_litellm/batches/test_batch_utils.py::test_handle_completed_vertex_batch_computes_cost_usage_and_models (assert 3.9375e-05 == 7.875e-05) — a Vertex batch cost assertion that is also red on other open PRs against the same base, unrelated to this change.

Also filed the missing bug report: #37430, with an e2e reproduction. The Proof section has been re-captured against the current tip (before at the merge base 423b791, after at ae54fca) rather than the older revision it described.

@greptileai

[CNSeniorious000]

Confirming the remaining red check is not from this PR: the target branch's own head is failing the same test.

• litellm_internal_staging @ 8941f2a — misc / Run tests failed: FAILED tests/test_litellm/batches/test_batch_utils.py::test_handle_completed_vertex_batch_computes_cost_usage_and_models - assert 3.9375e-05 == 7.875e-05
• this PR @ ae54fca — same job, same test, same numbers

The observed value is exactly half the expected one, which looks like batch pricing being applied where the assertion still expects list price — nothing this PR touches (it only adds an elif branch in langfuse_otel.py). Every other check on this PR is green (67/68).

Happy to open a separate PR for that batch-cost assertion if it's useful — just say the word and I'll dig into which side is wrong.

[CNSeniorious000]

Fixed in e6cd02b — the redaction gap was in redact_messages.py, not the serialization side. Both _redact_responses_api_output and _redact_responses_api_output_dict now redact custom_tool_call.input to redacted-by-litellm, mirroring the existing function_call.arguments branches, with regression tests for the dict and object forms.

[CNSeniorious000]

Friendly ping for review — this PR is ready from our side:

• One minimal change (folds custom_tool_call into the existing function_call branch in langfuse_otel.py), so Langfuse no longer logs output: null for custom tool calls ([Bug]: Langfuse logs output: null for Responses API custom (freeform) tool calls #37430 has the e2e repro)
• The redaction follow-up raised by @veria-ai is fixed in e6cd02b — custom_tool_call.input is now redacted in redact_messages.py alongside function_call.arguments, with regression tests for both the dict and object forms
• Latest commit is fully green across all checks (the earlier red check was the pre-existing test_handle_completed_vertex_batch_computes_cost_usage_and_models failure on the target branch itself)

Would love a maintainer's eyes when someone gets a chance. Happy to split the redaction fix into its own PR if that's preferred.

[CNSeniorious000]

Synced with staging and preserved safe JSON parsing. Updated redaction coverage and removed redundant comments. Please review the latest commit. @greptileai

[CNSeniorious000]

@cursor review the current commit, including the staging conflict resolution, custom tool output serialization, and both message-redaction paths

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8926f64. Configure here.}