Skip to content

fix(langfuse): preserve Responses tool namespaces - #41401

Open
CNSeniorious000 wants to merge 1 commit into
BerriAI:mainfrom
CNSeniorious000:fix-langfuse-tool-namespace
Open

CNSeniorious000 wants to merge 1 commit into
BerriAI:mainfrom
CNSeniorious000:fix-langfuse-tool-namespace

Conversation

@CNSeniorious000

@CNSeniorious000 CNSeniorious000 commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Langfuse loses tool namespaces present in Responses API output

How it solves it:

  • Copy non-null namespaces into logged function calls
  • Cover absent, named, and empty namespaces in regression tests

User Flow

Before: the application's logged tool call loses its namespace

  1. Send POST http://127.0.0.1:47381/v1/responses with the diagnostics.echo tool and "stream": true
  2. Receive a completed response containing name: "echo" and namespace: "diagnostics"
  3. Read the matching Langfuse generation's output and see name: "echo" without namespace

After: the logged tool call identifies the same namespace as the response

  1. Send POST http://127.0.0.1:47382/v1/responses with the diagnostics.echo tool and "stream": true
  2. Receive a completed response containing name: "echo" and namespace: "diagnostics"
  3. Read the matching Langfuse generation's output and see both name: "echo" and namespace: "diagnostics"

Relevant issues

Fixes #41400

Affected release

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

The full mapped test file passes: 38 passed. The added named and empty namespace cases fail against the base revision, while the absent case passes. Ruff check/format and type-discipline comparison against the base also pass

Screenshots / Proof of Fix

Captured against real model calls, with no mocks, on Python 3.13. The model was gpt-6-astra through an OpenAI-compatible endpoint. Only the private endpoint and credentials are replaced by environment variables below

# /tmp/namespace-proxy.yaml
model_list:
  - model_name: namespace-smoke
    litellm_params:
      model: openai/gpt-6-astra
      api_base: os.environ/NAMESPACE_UPSTREAM_BASE_URL
      api_key: os.environ/NAMESPACE_UPSTREAM_KEY
litellm_settings:
  callbacks:
    - langfuse_otel

Set NAMESPACE_UPSTREAM_BASE_URL to the provider's /v1 endpoint, NAMESPACE_UPSTREAM_KEY to its key, and LANGFUSE_HOST, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY to a Langfuse project. PYTHON below points to the Python environment with proxy and OTEL dependencies installed

Run each proxy in a separate terminal, from the corresponding checkout:

# Base checkout: a8979fe05466346d78ab4573c8a61b53a4dd9cf6
PYTHONPATH="$PWD" "$PYTHON" litellm/proxy/proxy_cli.py --config /tmp/namespace-proxy.yaml --port 47381 --host 127.0.0.1

# Fix checkout: c7e863e7be695d8af55b1e3bdd839fe51e142a16
PYTHONPATH="$PWD" "$PYTHON" litellm/proxy/proxy_cli.py --config /tmp/namespace-proxy.yaml --port 47382 --host 127.0.0.1

Shared request and capture commands (Bash, curl, jq):

cat > /tmp/namespace-request.json <<'JSON'
{
  "model": "namespace-smoke",
  "input": "Call diagnostics.echo with value namespace-smoke.",
  "tools": [{
    "type": "namespace",
    "name": "diagnostics",
    "description": "Smoke test",
    "tools": [{
      "type": "function",
      "name": "echo",
      "description": "Return the supplied value",
      "parameters": {
        "type": "object",
        "properties": {"value": {"type": "string"}},
        "required": ["value"],
        "additionalProperties": false
      },
      "strict": true
    }]
  }],
  "tool_choice": "required",
  "reasoning": {"effort": "low"},
  "max_output_tokens": 300,
  "store": false
}
JSON

run_smoke() {
  local port="$1" stream="$2" session="$3" label="$4"
  jq --argjson stream "$stream" --arg session "$session" --arg label "$label" \
    '. + {stream: $stream, litellm_metadata: {session_id: $session, trace_name: $label}}' \
    /tmp/namespace-request.json > "/tmp/$label-request.json"
  curl --silent --show-error --fail-with-body --max-time 150 \
    "http://127.0.0.1:$port/v1/responses" \
    -H 'Content-Type: application/json' \
    --data-binary "@/tmp/$label-request.json" -o "/tmp/$label-response.txt"
  if [ "$stream" = true ]; then
    jq -Rc 'select(startswith("data: ")) | .[6:] | fromjson? |
      select(.type == "response.completed") | .response.output[] |
      select(.type == "function_call") | {name, namespace, call_id}' "/tmp/$label-response.txt"
  else
    jq -c '.output[] | select(.type == "function_call") |
      {name, namespace, call_id}' "/tmp/$label-response.txt"
  fi
}

read_log() {
  curl --silent --show-error --fail-with-body \
    -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
    "$LANGFUSE_HOST/api/public/observations/$1" |
    jq -c '.output | if type == "string" then fromjson else . end | .[] |
      select(.type == "function_call") |
      {name, namespace_present: has("namespace"), namespace, call_id}'
}

Wait for export before reading the observation. The observation IDs below are from this capture. On a fresh run, find its trace with GET $LANGFUSE_HOST/api/public/traces?sessionId=<session>&limit=10, then use the matching generation's observation ID. litellm_metadata keeps this session association on the proxy being tested

The outputs below are the complete outputs of the jq projections shown above

Before (a8979fe)

Streaming

  1. Send the request and inspect the returned tool call

    run_smoke 47381 true db629521-fb40-491b-a642-81daf0beb0f8 proxy-before-stream
    {"name":"echo","namespace":"diagnostics","call_id":"call_k61isHCQAfXLKhxsaBReZSuY"}
  2. Read the same call from Langfuse

    read_log bea2dc96daa19991
    {"name":"echo","namespace_present":false,"namespace":null,"call_id":"call_k61isHCQAfXLKhxsaBReZSuY"}

Non-streaming

  1. Send the request and inspect the returned tool call

    run_smoke 47381 false cfae92b5-a201-4d2d-9750-6524087d6062 proxy-before-json
    {"name":"echo","namespace":"diagnostics","call_id":"call_LIB2OnWL1bcAAENU33Fg4bMp"}
  2. Read the same call from Langfuse

    read_log a159bb01cf751a04
    {"name":"echo","namespace_present":false,"namespace":null,"call_id":"call_LIB2OnWL1bcAAENU33Fg4bMp"}

After (c7e863e)

Streaming

  1. Send the request and inspect the returned tool call

    run_smoke 47382 true 09b4e387-71f2-4ddc-80c9-ee9e5fa2e751 proxy-after-stream
    {"name":"echo","namespace":"diagnostics","call_id":"call_ueLXXIOPkoPEgJ4SizZjgccS"}
  2. Read the same call from Langfuse

    read_log 16e7613c8fc8277c
    {"name":"echo","namespace_present":true,"namespace":"diagnostics","call_id":"call_ueLXXIOPkoPEgJ4SizZjgccS"}

Non-streaming

  1. Send the request and inspect the returned tool call

    run_smoke 47382 false cbf26433-2772-44db-bf3a-368b14de5d99 proxy-after-json
    {"name":"echo","namespace":"diagnostics","call_id":"call_OcKZFo6fiNL1LfieVnLdE8sS"}
  2. Read the same call from Langfuse

    read_log aacf184da560e545
    {"name":"echo","namespace_present":true,"namespace":"diagnostics","call_id":"call_OcKZFo6fiNL1LfieVnLdE8sS"}

Type

Bug Fix

Caveats (if any)

Low

Final Attestation

  • The tests check the right things, including namespace edge cases, and prevent this logged-namespace regression

@codspeed

codspeed Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing CNSeniorious000:fix-langfuse-tool-namespace (423b6ba) with main (c877e0f)

Open in CodSpeed

@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@CNSeniorious000
CNSeniorious000 marked this pull request as ready for review October 7, 2026 19:33
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Adds namespace field to tool call logging.

The namespace fix looks correct, but the no-mutation requirement must be satisfied before merging

Findings

  1. P2 Dictionary update breaks coding rules ▶

Summary

Preserves non-null Responses tool namespaces in Langfuse output, including empty strings

  • Extends the existing test with null, named, and empty namespaces
  • The added dictionary mutation must be rewritten to satisfy the coding guide

Reviews (1) · Last reviewed commit: "fix(langfuse): preserve Responses tool n..." · Reviewed by Greptile

"arguments": arguments_obj,
}
if (namespace := getattr(item, "namespace", None)) is not None:
langfuse_tool_call["namespace"] = namespace

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Dictionary update breaks coding rules

The new assignment mutates langfuse_tool_call, violating CLAUDE.md’s no-mutation directive for updated code. Build the optional field inside the dictionary expression before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@CNSeniorious000
CNSeniorious000 force-pushed the fix-langfuse-tool-namespace branch from c7e863e to 423b6ba Compare October 7, 2026 19:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Responses tool namespaces missing from Langfuse OTEL output

1 participant