Skip to content

fix(langchain): ensure llm spans are created for sync cases#3094

Closed
droidnxs wants to merge 1 commit intotraceloop:mainfrom
droidnxs:droidnxs/fix-langchain-missing-llm-spans
Closed

fix(langchain): ensure llm spans are created for sync cases#3094
droidnxs wants to merge 1 commit intotraceloop:mainfrom
droidnxs:droidnxs/fix-langchain-missing-llm-spans

Conversation

@droidnxs
Copy link
Copy Markdown

@droidnxs droidnxs commented Jul 7, 2025

Continuation of #2805

  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • (If applicable) I have updated the documentation accordingly.

Fixes #2271

Reproducible code from bug report

Courtesy #2271. Thanks @jemo21k!
Expected behaviour: LLM span should be created.

Before

No openai.chat span.
image

After

openai.chat span is present.
image

sample_app/langgraph_example

Expected behaviour: No change. Additional LLM span should not be created since ChatOpenAI callback already creates one.

Before

image

After

image

Important

Fixes LLM span creation for synchronous cases in Langchain and adds tests to verify behavior.

  • Behavior:
    • Ensures LLM spans are created for synchronous cases in Langchain by modifying _create_llm_span() in callback_handler.py.
    • Attaches context to suppress downstream instrumentation in __call__() in __init__.py and _create_llm_span() in callback_handler.py.
  • Tests:
    • Adds tests in test_langgraph.py to verify span creation for sync and async cases.
    • Adds VCR cassettes in tests/cassettes/test_langgraph/ to mock OpenAI API responses.
    • Updates test_langchain_metrics.py to include metrics tests for Langgraph.
  • Misc:
    • Updates pyproject.toml to include langgraph dependency.

This description was created by Ellipsis for 363a42a. You can customize this summary. It will automatically update as commits are pushed.

Copy link
Copy Markdown
Contributor

@ellipsis-dev ellipsis-dev Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Changes requested ❌

Reviewed everything up to 363a42a in 1 minute and 59 seconds. Click for details.
  • Reviewed 990 lines of code in 11 files
  • Skipped 1 files when reviewing.
  • Skipped posting 3 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/callback_handler.py:254
  • Draft comment:
    Replace 'if generation.message.content is str:' with an isinstance check (e.g. isinstance(generation.message.content, str)) to correctly verify the type.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/__init__.py:240
  • Draft comment:
    Typo: Consider changing "this should helps as a fallback" to "this should help as a fallback".
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment is technically correct about the grammar, it's an extremely minor issue in a comment. Comments are documentation, not code, and this small grammatical error doesn't impact understanding. The rules say not to make comments that are obvious or unimportant. The grammar error could be seen as unprofessional, and fixing it would improve code quality slightly. Bad grammar in comments could set a bad precedent. While professional code is important, this is an extremely minor issue that doesn't impact functionality or readability. The meaning is perfectly clear despite the small grammar error. Delete this comment as it's too minor and doesn't meaningfully improve the code quality.
3. packages/sample-app/sample_app/langgraph_openai.py:20
  • Draft comment:
    Possible typo: The model is set to "gpt-4o". Did you mean "gpt-4"?
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_SBbZH21UJWkc3poP

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.


# In legacy chains like LLMChain, suppressing model instrumentations
# within create_llm_span doesn't work, so this should helps as a fallback
context_api.attach(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider capturing the token returned by context_api.attach and detaching it after the wrapped call to limit the suppression flag's scope. This helps avoid unintended propagation in subsequent calls.

def calculate(state: State):
request = state["request"]
completion = openai_client.chat.completions.create(
model="gpt-4o",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo suggestion: The model string "gpt-4o" on this line might be a typographical error. If it was intended to be "gpt-4", please update accordingly.

Suggested change
model="gpt-4o",
model="gpt-4",

Copy link
Copy Markdown
Member

@nirga nirga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@droidnxs can you rebase? we had a huge PR that was waiting for a long time to get merged so it might broke some of this PR - lmk if you need help

@obs-gh-abhishekrao
Copy link
Copy Markdown
Contributor

Hi @nirga, I'm running busy with some critical work this week and next. If you could pick it up, that'll be really helpful.

@ronensc
Copy link
Copy Markdown
Collaborator

ronensc commented Jul 29, 2025

@obs-gh-abhishekrao Thanks for your work! I can pick this up. I'll rebase the commits, resolve the conflicts, and submit a new PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: OpenAI Requests Not Traced When Sent from a LangGraph Node

4 participants