Skip to content

feat: add OpenAI Agents SDK support to OpenInference mapper - #366

Draft
liramon2 wants to merge 1 commit into
strands-agents:mainfrom
liramon2:openai-openinference
Draft

feat: add OpenAI Agents SDK support to OpenInference mapper#366
liramon2 wants to merge 1 commit into
strands-agents:mainfrom
liramon2:openai-openinference

Conversation

@liramon2

@liramon2 liramon2 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Description

Add OpenAI agents support to OpenInference mapper.

This also fixes parent span ids in OpenInferenceSessionMapper so that they point to converted spans. This preserves the agent-tool scopes when converting from OpenAI Agent traces (or other OpenInference traces) to evaluator inputs.

Related Issues

#364

Documentation PR

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions github-actions Bot added enhancement New feature or request area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL labels Aug 12, 2026
@liramon2
liramon2 deployed to auto-approve August 12, 2026 21:35 — with GitHub Actions Active
@liramon2

Copy link
Copy Markdown
Contributor Author

@strandly-the-agent Review this PR. Consolidate findings into a single comment.

@strandly-the-agent

Copy link
Copy Markdown

Reviewed at bc6422d (base 1f752dc) with a multi-pass agentic review — correctness, adversarial, test-quality, API/design, LLM-context and scope passes, each a fresh-context run, then de-duped under a reachability gate. Consolidated into one comment as you asked.

Verdict: changes-requested. The producer works on the two shipped fixtures, but _normalize_openai_agents_trace reads three hard-coded attribute paths, and I broke all three against the real instrumentor's own source (v2.0.0, the version your ADOT fixture came from): multi-turn chats get a mismatched (prompt, response) pair, reasoning models get the chain-of-thought scored as the answer, and multimodal input drops the agent span. Separately, your own handoff fixture loses the coordinator and misattributes its handoff call to the agent that received it.

✅/🔴 Verified
Mapper suite 385 passed at bc6422d (main: 360) · extractors/types/detectors 235 passed
Hunk D (bridge_parent_gaps) is correct and is a genuine prerequisite, not a drive-by — don't split it out
Hunk F is inert for langchain/smolagents/claude (0/9 AGENT spans across all 5 existing fixtures carry llm.tools.*; no upstream instrumentor emits them on non-LLM spans)
🔴 5 failure modes reproduced in _normalize_openai_agents_trace; 3 confirmed against the instrumentor source
🔴 ADOT fixture: 3 raw AGENT spans → 1 AgentInvocationSpan; handoff tool misattributed
🟡 10 of 22 mutants survive all 385 tests; hunk D has zero tests

I patched every finding below locally and re-ran: 385 + 235 still pass, so these are cheap fixes rather than a redesign.


🔴 1 — _normalize_openai_agents_trace's hard-coded attribute paths (5 reproduced failure modes)

openinference_session_mapper.py:307 reads llm.input_messages.1.message.content; :320-322 reads llm.output_messages.0.... I downloaded openinference-instrumentation-openai-agents==2.0.0 and read _processor.py rather than guessing:

  • _get_attributes_from_input(obj, msg_idx: int = 1) (line 282) with enumerate(obj, msg_idx) (line 284) — the Responses-API input list is numbered from 1, so llm.input_messages.1 is the first item of the input list.
  • _get_attributes_from_chat_completions_message_dicts(..., msg_idx: int = 0) (line 513), reached from GenerationSpanData (line 186 → 488) — that path numbers from 0.
  • _get_attributes_from_response_output (line 684): a reasoning item occupies msg_idx and then increments it (lines 707-714), pushing the real answer to index 1.
# Manifestation Who hits it Result
a Multi-turn chat — index 1 is the oldest user message any agent using result.to_input_list() or Session/SQLiteSession (the documented multi-turn pattern) user_prompt='What is 15 multiplied by 37?' paired with agent_response='555 divided by 5 is 111.' — a judge scores the answer against the wrong question
b Reasoning models — summary sits at output index 0 gpt-5/o-series with reasoning.summary="auto" on instrumentor v2.0.0 agent_response='The user wants 15*37. Multiply: 15*37=555.' — the scratch-pad is scored as the answer. With encrypted reasoning and no summary, the AGENT span is dropped entirely
c Multimodal / structured content — instrumentor emits contents.0.message_content.text and no scalar .content the documented way to send images/files AgentInvocationSpan count = 0 — span silently dropped. Note _normalize_smolagents_llm_attrs (:176) already has this singular→plural fallback
d Chat-completions path numbers input from 0 LiteLLM / Azure / any OpenAI-compatible endpoint via the SDK index 1 is the second message
e Empty-string or tool-call-only final turn — guards are if user_prompt: / if agent_response:, then hunk E requires output.value tool_use_behavior="stop_on_first_tool", a max-turns cut-off, a guardrail trip AgentInvocationSpan count = 0 — exactly the runs an eval most wants to see

All five are silent — no warning, and the eval result comes back plausible.

This fix resolves a–d (verified; e needs a policy call from you, not a path fix — see the Questions):

_MSG_IDX_RE = re.compile(r"^llm\.(?:in|out)put_messages\.(\d+)\.")

@classmethod
def _message_indices(cls, attrs: dict, prefix: str) -> list[int]:
    """Descending message indices actually present under prefix (no contiguity assumed)."""
    out = set()
    for key in attrs:
        if key.startswith(prefix) and (m := cls._MSG_IDX_RE.match(key)):
            out.add(int(m.group(1)))
    return sorted(out, reverse=True)

@classmethod
def _last_user_message(cls, attrs: dict) -> str | None:
    """Text of the highest-indexed user input message, in either content shape."""
    for idx in cls._message_indices(attrs, "llm.input_messages."):
        base = f"llm.input_messages.{idx}.message"
        if attrs.get(f"{base}.role") != "user":
            continue
        if text := attrs.get(f"{base}.content") or attrs.get(f"{base}.contents.0.message_content.text"):
            return text
    return None

@classmethod
def _last_assistant_text(cls, attrs: dict) -> str | None:
    """Text of the highest-indexed non-reasoning assistant output message."""
    for idx in cls._message_indices(attrs, "llm.output_messages."):
        base = f"llm.output_messages.{idx}.message"
        if attrs.get(f"{base}.contents.0.message_content.type") == "reasoning":
            continue
        if text := attrs.get(f"{base}.contents.0.message_content.text") or attrs.get(f"{base}.content"):
            return text
    return None

then user_prompt = self._last_user_message(first_attrs) and agent_response = self._last_assistant_text(last_attrs). Deriving indices from the keys also removes the contiguity assumption in the llm.tools.{idx} copy loop at :313-316.

🔴 2 — multi-agent handoff: the coordinator disappears and its handoff call is misattributed to the agent that received it

Run on your own openai_agents_openinference_adot_spans.json:

raw AGENT spans: 3 -> ['coordinator', 'Agent workflow', 'math_specialist']
surviving AgentInvocationSpans: 1
  span_id=2cbdd8e10afa264f
    user_prompt="What is 42 * 7, and what's the weather in Seattle?"
    available_tools=['add_numbers', 'divide_numbers', 'multiply_numbers']
  TOOL 'handoff to math_specialist' parent=None agent_span_id=2cbdd8e10afa264f
  TOOL 'multiply_numbers'           parent=2cbdd8e10afa264f agent_span_id=2cbdd8e10afa264f
  • "Agent workflow" is dropped: the walk (:289-299) recurses only into CHAIN children and stops at nested AGENT children, so it collects no LLM spans and never gets input.value.
  • coordinator is dropped: its LLM span emits only tool_calls (the handoff decision) and no text, so output.value is never set and hunk E (:452-458) rejects it. Its declared transfer_to_math_specialist / transfer_to_research_specialist schemas are lost with it.
  • math_specialist survives carrying the original top-level question, so the trace reads as one agent that was asked the whole thing and answered it. The delegation is invisible.
  • The handoff TOOL span gets parent=None from bridge_parent_gaps (its real ancestor was dropped), and then Trace.model_post_init's orphan fallback (types/trace.py:202-206) assigns agent_span_id = math_specialistattributing the coordinator's delegation to the agent that received it.

Concretely, a ToolSelectionAccuracy judge is asked to justify handoff to math_specialist({}) while the "Available tool-calls" list it was shown is add_numbers/divide_numbers/multiply_numbers — the tool under evaluation isn't in the candidate set it was given.

For contrast, the Claude Agent SDK producer represents delegation legibly, as Agent({'description': …, 'prompt': …}) with the subagent's real answer as tool_result.

The misattribution is wrong regardless of intent, so it needs fixing either way; whether the coordinator should survive is a design call I've put in the Questions. Cheapest correct step: don't let an orphaned tool span inherit agent_span_id from an unrelated agent — leave it None and let the extractors treat it as unowned.

Worth noting test_math_specialist_agent_span currently asserts only "at least one agent span" matching 42 or weather, so it encodes this collapse as expected. There's no agent-count assertion for this fixture (the live one has test_one_agent_span_per_trace).

🟡 3 — the normalizer runs outside every try/except, so one bad span can lose the whole session (including other producers' traces)

map_to_session:137-140 calls the normalizer before any span reaches _build_trace's per-span try/except Exception + logger.warning (:331, :344-345). Reproduced — a healthy LangChain trace batched with one awkward openai_agents trace:

healthy LangChain trace alone -> ['t-langchain']
TypeError ESCAPED map_to_session: '<' not supported between instances of 'int' and 'str'
  -> 0 traces returned: the healthy LangChain trace is destroyed too
attributes=None variant: AttributeError ESCAPED map_to_session: 'NoneType' object has no attribute 'get'

Triggers: llm_spans.sort(key=lambda s: s.get("start_time", 0)) (:303) on heterogeneous/None start_time, and attributes: None (:282/:293).

Honest reachability, which is why this is 🟡 and not 🔴: I could not reach it through your own CloudWatchLogsParser — it drops records lacking startTimeUnixNano, so values come out uniform. But SessionMapper.parse_timestamp (session_mapper.py:77) explicitly documents start_time as possibly str/datetime/None, so the class's declared input contract permits the crashing input — and the correct idiom already exists next door in adk_otel_session_mapper.py:194. Every other per-span quirk here degrades to a warning; this one takes unrelated producers down with it.

llm_spans.sort(key=lambda s: self.parse_timestamp(s.get("start_time")))

plus wrapping the call:

try:
    self._normalize_openai_agents_trace(trace_spans)
except Exception as e:
    logger.warning("Failed to normalize OpenAI Agents trace: %s", e)

With both applied the batch above returns ['t-langchain'] and logs a warning.

🟡 4 — test gaps: 10 of 22 mutants survive all 385 tests, and hunk D is completely unpinned

Deleting the bridge_parent_gaps call at :349 outright leaves 123 passed in the mapper file and 235 passed in extractors/types/detectors. The correct behaviour it provides (3 spans re-parented onto math_specialist) is real and worth pinning.

Other survivors: input taken from the last LLM descendant instead of the first · the entire llm.tools.* copy made a no-op · the .message.content output fallback deleted · hunk F's _trace_tools_map fallback deleted · the and "name" not in tool_info guard dropped · the or tool_info.get("parameters") fallback dropped · the added ValueError removed · the attrs.get("input.value") skip guard removed · the if user_prompt: guard removed.

Also: hunk F is pinned by exactly one test (test_openai_agent_span_conversion), which hand-builds an AGENT span carrying its own llm.tools.0 — a shape no real producer emits. 5 of the 10 TestOpenAIAgentsScopeSupport tests only assert _is_*_span(...) is True/False with no mapping exercised. And test_handoff_tool_span_present asserts len(tool_spans) >= 1 without filtering by name, so it passes even if the handoff span specifically is dropped.

Two that pass today and close the D / agent-count gaps:

def test_math_specialist_children_reparented(self, openai_agents_adot_session):
    all_spans = [s for t in openai_agents_adot_session.traces for s in t.spans]
    specialist = next(s for s in all_spans if isinstance(s, AgentInvocationSpan) and "42" in s.user_prompt)
    multiply = next(s for s in all_spans if isinstance(s, ToolExecutionSpan) and s.tool_call.name == "multiply_numbers")
    inferences = [
        s for s in all_spans
        if isinstance(s, InferenceSpan) and s.span_info.parent_span_id == specialist.span_info.span_id
    ]
    assert multiply.span_info.parent_span_id == specialist.span_info.span_id
    assert len(inferences) == 2

def test_adot_agent_span_count_is_one(self, openai_agents_adot_session):
    for trace in openai_agents_adot_session.traces:
        assert len([s for s in trace.spans if isinstance(s, AgentInvocationSpan)]) == 1

Plus one that fails today and pins the appendix-1 regression: a make_span AGENT with "llm.tools.0.tool.json_schema": "42" should still yield 1 agent span with available_tools == [].

Questions

  1. Is the coordinator's disappearance the intended representation of a handoff? An argument exists for "the agent that produced the final answer is the trace" — but then math_specialist inheriting the top-level question and owning the handoff call reads as one agent doing everything, and the coordinator's transfer_to_* tools vanish. Would you rather a tool-call-only turn counted as a real agent invocation? (blocking — it decides whether 🔴 2 is a bug or a documented limitation, and whether manifestation (e) above needs the same treatment.)
  2. feat(mappers): add OpenAI Agents OTel session mapper #365 vs feat: add OpenAI Agents SDK support to OpenInference mapper #366 — two techniques for one problem. feat(mappers): add OpenAI Agents OTel session mapper #365 attaches tools directly to the converted AgentInvocationSpan (openai_agents_otel_session_mapper.py:61-68), whereas here hunk C fabricates llm.tools.* onto the raw AGENT span (:313-316) so hunk F (:701-706) can read it back. The input.value/output.value half of C is load-bearing for the :452-458 gate and clearly earns its place, but would the tool half be simpler as the feat(mappers): add OpenAI Agents OTel session mapper #365 approach? Both PRs also make the same parent-span-id fix in parallel — is one meant to land first? (non-blocking)
  3. Following the feat: add support for Claude agents to OpenInference mapper #340feat: add Claude Agents OpenInference integration tests #353 precedent, is there a tracking issue for tests_integ/test_openai_agents_*? I couldn't find one. (non-blocking)
Appendix — non-blocking (7)
  1. :889 if "function" in tool_info and … assumes a dict. A llm.tools.N.tool.json_schema decoding to a JSON scalar raises TypeError, which isn't in the except (json.JSONDecodeError, AttributeError, ValueError) at :898; it escapes and _build_trace's broad except Exception drops the whole span, where main skipped just that one tool. Strictly contrived — I checked openai_agents v1.6.1/v2.0.0, vendored langchain (which gates LLM_TOOLS to LLM runs) and claude_agent_sdk (emits no llm.tools.* at all): no real instrumentor can emit it. But it's a regression vs main and the fix is one word: isinstance(tool_info, dict) and ….
  2. The descendant walk at :289-299 has no visited set, unlike bridge_parent_gaps (utils.py:298-301). A duplicate span_id closing a cycle hangs the process with unbounded llm_spans growth (confirmed, >5s with no termination); self-parented and diamond variants terminate fine. Contrived, one-line fix.
  3. Hunk C copies tool schemas from the earliest LLM span only, so a tool list that changes across turns loses entries (crafted case: 1 tool instead of 3). Reachable via FunctionTool.is_enabled per-turn filtering or MCP with cache_tools_list=False. Uncommon.
  4. The normalizer never checks scopemap_to_session:139 gates on "any span in the group is openai_agents" but hands the walk the whole group. Sibling spans are safe (I checked: a smolagents AGENT span sharing a trace_id is not mutated), so this needs a non-openai AGENT span to be an ancestor of openai CHAIN/LLM spans. Contrived, but a scope check in the loop would make the intent explicit.
  5. ⚪ Hunk F is safe for the 3 existing producers by upstream construction, not by a guard here — worth a comment saying so, since nothing in the suite would catch a future instrumentor that starts attaching llm.tools.* to AGENT spans.
  6. ⚪ Fixture provenance skew: the live fixture is instrumentor v1.6.1, the ADOT one v2.0.0. Worth a word on whether that's deliberate.
  7. ⚪ Correcting myself on a couple of things a reviewer might otherwise raise, in case they come up: the in-place mutation of caller span dicts is pre-existing in this class since fix: route smolagents OpenInference spans to OpenInferenceSessionMapper #308 (main:191, :204, :256, :259) and is observably idempotent, so it isn't this PR's problem; and the _adot_ fixture is legitimately ADOT-captured (aws.genai.*, PlatformType: AWS::BedrockAgentCore) — the existing claude_adot fixture has the same flat shape, so "it doesn't exercise _parse_adot_output" isn't a fault of this PR either.

Three pre-existing items are out of scope here but might deserve their own issues — happy to file them if useful: the orphan-fallback in types/trace.py:202-206 silently attributing any unowned tool span to the sole surviving agent (affects every mapper); the is_langchain dedup exemption at :351-357 having had no test since #340; and no mapper documenting the AgentInvocationSpan-per-Trace expectation that trace_extractor and root_cause_analyzer each interpret differently.

Automated multi-pass review by strandly-the-agent. Every 🔴/🟡 above has a repro I ran at bc6422d, and I applied all the suggested fixes locally to confirm they work and keep 385 + 235 green — but I'm an experimental agent, so please treat this as work for a human to approve rather than a verdict. Happy to dig into any of it further.

@poshinchen

Copy link
Copy Markdown
Contributor

@strandly-the-agent Here are the instrumentation:

  1. "https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai-agents",
  2. "https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai",
  3. "https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-openai-agents",
  4. "https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-openai"

Here are the current findings:

For OpenAI agents SDK + traceloop instrumentation:

  • It mostly follows GenAI conventions
  • The instrumentation does not populate agent spans' available_tools . Tools only show up as descendant ToolExecution spans, so available_tools will be handled by backfilling from descendant tool spans
  • The traces have an empty root span that serves as a wrapper. Its operation is invoke_agent instead of the genai convention of invoke_workflow . These should be stripped to avoid having an empty agent span.
  • Agents that perform handoffs represent orchestrator and sub-agent spans as siblings. Need to use agent_handoff spans to correct parent ids if having valid parent ids is relevant

For OpenAI agents SDK + OpenInference instrumentation:

  • It mostly follows openinference conventions
  • input.value and output.value are not agent span attributes. They are instead on descendant LLM spans. So they are handled with a walk down to those spans and looking into llm.input_messages.N.message.content.0.message_content.text and llm.output_messages.N.message.contents.0.message_content.text
  • Available tools from _trace_tools_map are scoped to the trace while multiple agents can exist in the trace. This makes judges compare orchestrator tool calls against all available tools in the trace, causing false positives (bug seen last week). This can be handled by copying LLM tool schemas llm.tools.N.tool.json_schema onto AGENT spans and extracting them into available_tools . Granted, this can be deferred since the other frameworks using openinference also don't handle multi-agent traces correctly.

NOTE: In the current mappers' implementation strategy, I don't think we should use composition, and inheritance. And the mappers are grouped by instrumentation frameworks.

I need you to:

  1. verify the above claims,
  2. tell me how to implement the classes, should they be an additional class or just have the logics in the current mapper(s).
  3. I don't think we need inheritance for now. Tell me the suggestions

@strandly-the-agent

Copy link
Copy Markdown

Verified all seven claims against the instrumentation sources (openinference-instrumentation-openai-agents 2.0.0, opentelemetry-instrumentation-openai-agents 0.62.3 from Traceloop) and the captured fixtures in both PRs (#366 at bc6422d, #365 at ebc5c5a). Six hold; three need precision that changes the remedy.

# Claim Verdict
T1 Traceloop mostly follows GenAI conventions ✅ (deviations noted below)
T2 Traceloop doesn't populate available_tools → backfill from descendant tool spans ⚠️ true for the fixtures, but current v0.62.3 emits gen_ai.tool.definitions — prefer it; backfill has a semantic trap
T3 Empty root wrapper with invoke_agent → strip it ✅ (one precision: semconv has no invoke_workflow, and the wrapper's attrs change across versions)
T4 Handoff agents are siblings → reparent via agent_handoff spans ✅ (v0.62.3 adds a simpler signal: gen_ai.agent.handoff_parent on the sub-agent span)
O1 OpenInference mostly follows its conventions
O2 input/output live on descendant LLM spans → walk + read message paths ⚠️ mechanism right; the key paths as written are off, and index selection is the actual hard part
O3 _trace_tools_map is trace-scoped → multi-agent false positives; copy llm.tools.* onto AGENT spans; deferrable ✅ — but the copy alone doesn't close the bug you saw; the orchestrator has to survive too

Design answer (Q2/Q3): agree — no inheritance, and no new classes. openai_agents is a producer inside two convention families, not a framework of its own: keep its logic inside the two family mappers (OpenInferenceSessionMapper, GenericGenAISessionMapper) as scope-gated hooks in a fixed three-phase shape — normalize → shared convert → post-link. Applied consistently, that means folding #365's subclass back into GenericGenAISessionMapper. Details and a sketch below.

Traceloop verification (T1–T4)

T1 — GenAI conventions ✅. Fixture chat spans carry gen_ai.operation.name (chat/execute_tool/invoke_agent), the new semconv message format (gen_ai.input.messages/gen_ai.output.messages as role/parts JSON), gen_ai.usage.*, gen_ai.agent.name/id/description. Deviations to plan for: per-turn wrapper spans with gen_ai.operation.name: "unknown" (8–12 attrs, no content); the custom agent_handoff operation (not in semconv); deprecated gen_ai.system alongside gen_ai.provider.name.

T2 — ⚠️ version-dependent, and the backfill has a semantic trap.

  • In both feat(mappers): add OpenAI Agents OTel session mapper #365 fixtures the claim is exactly right: the only tool-ish attributes anywhere are gen_ai.tool.call.{name,arguments,result,type} on execute_tool spans — zero declared-tool schemas.
  • But the current release emits the real declared list: v0.62.3's _end_generation_span sets gen_ai.tool.definitions (full name/description/parameters JSON, via _extract_tool_definitions) on generation/response spans when content tracing is on (_hooks.py:1021-1028; runs for GenerationSpanData/ResponseSpanData, :800-803). The feat(mappers): add OpenAI Agents OTel session mapper #365 fixtures simply predate it.
  • So the remedy should be: prefer gen_ai.tool.definitions when present, fall back to backfill. Two reasons the backfill alone is worth improving: (i) descendant ToolExecutionSpans are tools used, not tools available — a tool-selection judge sees a shrunken candidate set with the un-chosen options missing, which is the mirror image of the false-positive bug you saw on the OpenInference side (candidate set too big there, too small here); (ii) feat(mappers): add OpenAI Agents OTel session mapper #365's backfill builds ToolConfig(name=...) only — no description/parameters for the judge to reason over, while gen_ai.tool.definitions carries both.

T3 — ✅ confirmed in both fixtures. Root Agent workflow span with gen_ai.operation.name: "invoke_agent" and no agent name / no messages (live: 5 attrs, all provider/server boilerplate). Stripping is right, and #365's rule (no gen_ai.agent.name and no gen_ai.input.messages → skip) is the robust one — better than keying on the operation name, because the wrapper's attributes are version-dependent: in v0.62.3 the root gets traceloop.span.kind: workflow and no gen_ai.operation.name at all (on_trace_start, _hooks.py:676-688). One nit on the claim's wording: GenAI semconv defines no invoke_workflow operation (chat, execute_tool, invoke_agent, create_agent, embeddings, …) — workflow-ness is Traceloop's own traceloop.span.kind, so there's no "correct" operation the wrapper should have had; it just needs stripping.

T4 — ✅ confirmed. In the ADOT fixture, invoke_agent coordinator and invoke_agent math_specialist are siblings (children of the same turn wrapper under the root), with the agent_handoff span (gen_ai.handoff.from_agent: coordinator, gen_ai.handoff.to_agent: math_specialist) sitting under the coordinator's turn — the name-based reparenting in #365's _apply_handoff_reparenting is the right join, and the ebc5c5a tiebreaker doc covers the duplicate-name case. Two additions:

  • The agents-as-tools pattern needs no correction — in the live fixture, execute_tool ask_math_specialist → nested invoke_agent math_specialist nests properly already. Sibling-flattening is handoff-specific.
  • v0.62.3 also stamps gen_ai.agent.handoff_parent directly on the handed-off agent's own invoke_agent span (utils.py:14, applied in _start_agent_span via _reverse_handoffs_dict). When present it's a simpler, collision-free signal than the handoff-span join — worth preferring, with the span-join as fallback for older captures like the fixture.
OpenInference verification (O1–O3)

O2 — mechanism ✅, paths and indices need precision. Verified against the v2.0.0 instrumentor source and both #366 fixtures:

  • Input: plain-text messages carry the scalar llm.input_messages.N.message.content — that's the only input shape in both shipped fixtures. The contents.0.message_content.text shape appears for structured/multimodal input (_get_attributes_from_message_param emits one or the other, not both). The path as written in your comment (...message.content.0.message_content.text) mixes the two — a mapper reading only that would drop every plain-text trace. Read scalar first, contents.0...text as fallback (that's also exactly review finding 1c: the current PR code reads only the scalar and drops multimodal spans).
  • Output: contents.0.message_content.text on v2.0.0, scalar .message.content on v1.6.1 — both fallbacks needed (the PR does this part right).
  • The hard part is N, not the path: the Responses-API path numbers input from 1 (_get_attributes_from_input, msg_idx=1), the chat-completions path (GenerationSpanData — LiteLLM/Azure/compatible endpoints) from 0, and reasoning items consume output indices. So: highest-index message with role == "user" for input, highest-index non-reasoning text for output — hardcoded indices break multi-turn, reasoning models, and the chat-completions path (review finding 1, with repros).

O3 — ✅ verified, with one important limit. _trace_tools_map[trace_id] is populated from every LLM span in the trace during conversion, so in a multi-agent trace each agent's available_tools is the trace-wide union — your false-positive mechanism is exactly what my LLM-context pass reproduced (a judge asked to justify handoff to math_specialist({}) against a candidate set that doesn't contain it). Copying llm.tools.N.tool.json_schema onto AGENT spans fixes the scoping, but as shipped it doesn't close the bug you saw, for two reasons:

  • The orchestrator's AGENT span is currently dropped (its final LLM turn is tool-calls-only, so output.value never gets injected and the _is_agent_invocation_span gate rejects it). Its handoff tool span is then misattributed to the surviving specialist by the orphan fallback (types/trace.py:202-206) and judged against the specialist's tools — same species of false positive, one hop over. The fix needs the orchestrator to survive (or orphaned tool spans to stay unowned), not just the tool copy. That's review blocker 2 / Question 1 — it's the decision this hinges on.
  • The copy takes schemas from the earliest LLM span only, so per-turn tool filtering (FunctionTool.is_enabled, MCP without cache) loses entries. Deriving indices from keys and merging turns is cheap.
  • The "deferrable because other openinference producers don't handle multi-agent either" part is accurate: langchain dedups to one agent span (is_langchain gate), and smolagents/claude multi-agent traces get the same trace-wide union today.
Design recommendation (Q2/Q3): no new classes, no inheritance — three named phases inside the two family mappers

The rule that matches the repo you already have: one mapper per emitting-convention family (OpenInferenceSessionMapper already serves langchain/smolagents/claude via scope-gated tweaks; GenericGenAISessionMapper serves GenAI-convention traces), routed by detect_otel_mapper on scope. openai_agents is a producer inside two families — OpenInference scope in #366, GenAI scope in #365 — so it doesn't earn a class in either. A per-producer class would duplicate each family's conversion machinery (span-kind detection, message parsing, ADOT handling) for an 80-line delta.

Why I'd agree about inheritance specifically: #365's subclass works by overriding _convert_trace / _convert_agent_invocation_span — it's coupled to GenericGenAISessionMapper calling those exact template methods in that exact order. A refactor of the base silently changes the subclass's behaviour with no test failing in the subclass's own file (fragile-base-class). Scope-gated hooks keep the coupling visible in one place. Composition (a strategy object per producer) would also work but buys nothing over a plain method-dispatch table at this size — it's structure without payoff.

The shape I'd standardize (both mappers already approximate it):

  1. Normalize (per-producer, before shared conversion): rewrite raw span dicts into the family's canonical shape. _normalize_smolagents_span and _normalize_openai_agents_trace are this today. Registered in a dispatch table rather than a growing if-chain:
    # producer scope -> (granularity, hook)
    _PRODUCER_NORMALIZERS = {
        SCOPE_OPENINFERENCE_SMOLAGENTS: ("span", _normalize_smolagents_span),
        SCOPE_OPENINFERENCE_OPENAI_AGENTS: ("trace", _normalize_openai_agents_trace),
    }
  2. Convert (shared, producer-blind): one code path per family. Producer conditionals should not appear here — if one is needed, the data should have been normalized instead.
  3. Post-link (per-producer where needed, after conversion): structural fixes on converted spans — feat(mappers): add OpenAI Agents OTel session mapper #365's handoff reparenting and tools backfill live here; bridge_parent_gaps is the family-agnostic version and stays shared in utils.

Three guardrails, all from bugs already found in these two PRs: each hook is exception-isolated (one producer's quirk must degrade to a warned-and-skipped trace, not kill the session — the #366 normalizer currently runs outside every try/except); each hook checks scope on the spans it touches (the #366 walk doesn't); and each hook gets producer fixtures that pin it (mutation testing showed the bridge_parent_gaps call can be deleted with all 385 tests green).

Applied to the two PRs:

  • feat: add OpenAI Agents SDK support to OpenInference mapper #366: keep everything in OpenInferenceSessionMapper — no new class. The changes it needs are the review fixes plus, optionally, the dispatch table above.
  • feat(mappers): add OpenAI Agents OTel session mapper #365: the consistent no-inheritance answer is to fold OpenAIAgentsOtelSessionMapper into GenericGenAISessionMapper: wrapper-strip becomes a normalize/skip rule, handoff-reparent + tools-backfill become a post-link step, both gated on SCOPE_OPENAI_AGENTS, and detect_otel_mapper:140-141 routes to the generic mapper. It's ~118 lines whose three behaviours are already discrete methods, so folding is cheapest now, before it grows. Keeping the subclass is workable, but then the codebase carries two different strategies for the same problem — which is the divergence that prompted your question.

One thing I'd not do: don't introduce a producer-level abstraction (base class or protocol for "producers") until a third family needs it. Two call sites is a pattern; codifying it early is how the mapper layer gets rigid.

Verification by strandly-the-agent: claims checked against instrumentor sources (openinference 2.0.0, traceloop 0.62.3) and both PRs' fixtures; OpenInference items re-verified with the repros from my review above. As ever — my analysis to check, your call to make.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants