feat: add support for Claude agents to OpenInference mapper by liramon2 · Pull Request #340 · strands-agents/evals

liramon2 · 2026-07-31T21:43:13Z

Description

The evals framework currently cannot evaluate agents built with Claude Agent SDK, which may be instrumented with OpenInference and produce spans with the scope openinference.instrumentation.claude_agent_sdk. This PR modifies OpenInferenceSessionMapper to map Claude Agent spans into Sessions. It includes support for unified ADOT spans and in-memory spans.

This also includes producer agnostic fixes for the following OpenInference bugs:

If output.value.content on execute_tool spans contained a list of text blocks, it was converted into a repr string instead having its text joined. Similarly, if output.value itself was a JSON list of content blocks, it was stored as the raw JSON string without text extraction.
Agent spans with an error code but no output value were rejected from conversion instead of surfaced to the judge.

Related Issues

#329

Documentation PR

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

I ran hatch run prepare

Checklist

I have read the CONTRIBUTING document
I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
My change is focused and reasonably small; I have split unrelated work into separate PRs
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

liramon2 · 2026-07-31T21:43:41Z

@strandly-the-agent Review this PR. Consolidate your findings into a single comment.

liramon2 · 2026-08-01T00:32:07Z

@strandly-the-agent Review this PR. Consolidate your findings into a single comment. As for your questions:

Traces with multiple invoke agent spans should be supported for multi-agent invocations
claude_live_spans.json is a live capture with signatures mocked
The lack of input.value / output.value is a deliberate design decision by the instrumentation

liramon2 · 2026-08-03T01:20:00Z

@strandly-the-agent Review this PR. Consolidate your findings into a single comment. Finding B is out of the scope of this PR and tracked in #336

liramon2 · 2026-08-04T20:46:38Z

@strandly-the-agent Review this PR. Consolidate your findings into a single comment.

liramon2 · 2026-08-04T21:57:13Z

@strandly-the-agent Review this PR. Consolidate your findings into a single comment. Mutation 4 seems attached to the 336 bug. Flatten-to-text is the right contract, but it's speculative right now.

poshinchen

Make sure the description is updated!

Resolves one conflict in the OpenInference mapper's `AgentInvocationSpan` construction. Upstream's strands-agents#340 added `metadata=metadata` from `_extract_llm_metadata`; this branch added `system_prompt` so the skill catalog survives mapping. The two set different fields, so both are kept, taking upstream's real metadata over this branch's `{}` placeholder.

liramon2 requested a deployment to manual-approval July 31, 2026 21:43 — with GitHub Actions Waiting

liramon2 temporarily deployed to auto-approve July 31, 2026 21:43 — with GitHub Actions Inactive

github-actions Bot added the enhancement New feature or request label Jul 31, 2026

github-actions Bot added the area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL label Jul 31, 2026