Improve Strands Tool Error Capturing by TimPansino · Pull Request #1623 · newrelic/newrelic-python-agent

TimPansino · 2026-01-09T20:08:16Z

Overview

Add additional instrumentation to Strands Agents to capture errors in synchronous tool functions decorated with @tool.
Change coverage of all tools in Strands to track errors emitted from stream() instead of relying on the _tool_func attribute for better coverage.
Overhaul Strands Agents testing to better cover the entire matrix of tool types and invocation functions.

github-actions · 2026-01-09T20:09:43Z

✅MegaLinter analysis: Success

Descriptor	Linter	Files	Fixed	Elapsed time
✅ ACTION	actionlint	7		0.98s
✅ MARKDOWN	markdownlint	7	0	1.19s
✅ PYTHON	ruff	963	0	0.88s
✅ PYTHON	ruff-format	963	0	0.32s
✅ YAML	prettier	15	0	1.33s
✅ YAML	v8r	15		5.3s
✅ YAML	yamllint	15		0.65s

See detailed reports in MegaLinter artifacts

codecov-commenter · 2026-01-09T20:37:26Z

Codecov Report

❌ Patch coverage is 74.07407% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.73%. Comparing base (d5bb817) to head (0834599).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
newrelic/hooks/mlmodel_strands.py	73.07%	5 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1623      +/-   ##
==========================================
+ Coverage   81.70%   81.73%   +0.03%     
==========================================
  Files         209      209              
  Lines       24388    24412      +24     
  Branches     3866     3869       +3     
==========================================
+ Hits        19926    19953      +27     
+ Misses       3157     3154       -3     
  Partials     1305     1305

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

newrelic/hooks/mlmodel_strands.py

* Update output message timestamping. (#1627) * Update output message timestamping. * Fix LangChain tests. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Guard Azure Functions Utilization (#1632) * Add guard to azure functions utilization for crashes * Add regression test * Improve Strands Tool Error Capturing (#1623) * Overhaul Strands Agents testing * Add better error instrumentation to strands tools * Expand testing for strands tools * Add better guarding to register_tool instrumentation * Fix up test logic * Add tool_id to all strands errors * Add log message --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add support for BaseException instances as arguments to notice_error (#1571) * Update implementation of notice_error * Clean up test_notice_error file * Add tests for notice_error with exception instances --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * LangGraph Instrumentation Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * LangChain Agents Instrumentation Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Update tox versions for LangChain / LangGraph Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Move GeneratorProxy from Strands to common file Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * More verbose logging in validate_custom_events Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Improve prompt logging for mock openai server Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Tweak langchain test folder structure Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Update Chain tests Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * New Agent testing Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * New Tool testing Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Expand Test Matrixing Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Newly recorded responses for LangChain Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Patch incorrect super() call in GeneratorProxy Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Better entry point for agent exception testing * Update AgentObjectProxy to include transform() methods Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> * Update event counts in RunnableSequence tests * Reformatting to kwargs * Formatting * Remove storage of agent name on transaction * Instrument RunnableSequence.stream and astream * Add correct event counts * Guard metadata additions * Add alternate source for agent_name * Pin lower bound of langchain tests * Implement tee() and __copy__() for GeneratorProxy * Slight tweaks * Fixups. * Fix pyramid by pinning setuptools (#1647) * Bump the github_actions group across 1 directory with 7 updates (#1657) Bumps the github_actions group with 7 updates in the / directory: | Package | From | To | | --- | --- | --- | | [actions/checkout](https://github.com/actions/checkout) | `6.0.1` | `6.0.2` | | [actions/setup-python](https://github.com/actions/setup-python) | `6.1.0` | `6.2.0` | | [docker/login-action](https://github.com/docker/login-action) | `3.6.0` | `3.7.0` | | [docker/build-push-action](https://github.com/docker/build-push-action) | `6.18.0` | `6.19.1` | | [actions/attest-build-provenance](https://github.com/actions/attest-build-provenance) | `3.1.0` | `3.2.0` | | [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `7.2.0` | `7.3.0` | | [github/codeql-action](https://github.com/github/codeql-action) | `4.31.10` | `4.32.2` | Updates `actions/checkout` from 6.0.1 to 6.0.2 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@8e8c483...de0fac2) Updates `actions/setup-python` from 6.1.0 to 6.2.0 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@83679a8...a309ff8) Updates `docker/login-action` from 3.6.0 to 3.7.0 - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@5e57cd1...c94ce9f) Updates `docker/build-push-action` from 6.18.0 to 6.19.1 - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](docker/build-push-action@2634353...601a80b) Updates `actions/attest-build-provenance` from 3.1.0 to 3.2.0 - [Release notes](https://github.com/actions/attest-build-provenance/releases) - [Changelog](https://github.com/actions/attest-build-provenance/blob/main/RELEASE.md) - [Commits](actions/attest-build-provenance@00014ed...96278af) Updates `astral-sh/setup-uv` from 7.2.0 to 7.3.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@61cb8a9...eac588a) Updates `github/codeql-action` from 4.31.10 to 4.32.2 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@cdefb33...45cbd0c) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github_actions - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions - dependency-name: docker/login-action dependency-version: 3.7.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions - dependency-name: docker/build-push-action dependency-version: 6.19.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions - dependency-name: actions/attest-build-provenance dependency-version: 3.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions - dependency-name: astral-sh/setup-uv dependency-version: 7.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions - dependency-name: github/codeql-action dependency-version: 4.32.2 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github_actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Uma Annamalai <uannamalai@newrelic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Uma Annamalai <umaannamalai@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

TimPansino added 3 commits January 9, 2026 12:05

Overhaul Strands Agents testing

83bffef

Add better error instrumentation to strands tools

63b93ae

Expand testing for strands tools

f5de1c2

TimPansino requested a review from a team as a code owner January 9, 2026 20:08

Add better guarding to register_tool instrumentation

1bb468f

mergify bot added the tests-failing Tests failing in CI. label Jan 9, 2026

Merge branch 'main' into fix-strands-errors

ab0d55f

mergify bot removed the tests-failing Tests failing in CI. label Jan 9, 2026

Fix up test logic

758632b

mergify bot added the tests-failing Tests failing in CI. label Jan 13, 2026

umaannamalai reviewed Jan 13, 2026

View reviewed changes

newrelic/hooks/mlmodel_strands.py Outdated Show resolved Hide resolved

newrelic/hooks/mlmodel_strands.py Show resolved Hide resolved

mergify bot and others added 3 commits January 15, 2026 18:52

Merge branch 'main' into fix-strands-errors

d80a0d2

Add tool_id to all strands errors

1a70e1e

Add log message

89ef0b3

umaannamalai approved these changes Jan 22, 2026

View reviewed changes

Merge branch 'main' into fix-strands-errors

2108116

TimPansino enabled auto-merge (squash) January 22, 2026 20:02

Merge branch 'main' into fix-strands-errors

e685ad9

mergify bot added tests-failing Tests failing in CI. and removed tests-failing Tests failing in CI. labels Jan 22, 2026

Merge branch 'main' into fix-strands-errors

0834599

TimPansino merged commit a8b6bd8 into main Jan 22, 2026
105 of 108 checks passed

TimPansino deleted the fix-strands-errors branch January 22, 2026 21:13

mergify bot removed the tests-failing Tests failing in CI. label Jan 22, 2026

umaannamalai added this to the v11.3.0 milestone Jan 22, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Improve Strands Tool Error Capturing#1623

Improve Strands Tool Error Capturing#1623
TimPansino merged 12 commits intomainfrom
fix-strands-errors

TimPansino commented Jan 9, 2026

Uh oh!

github-actions bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jan 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

TimPansino commented Jan 9, 2026

Overview

Uh oh!

github-actions bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅MegaLinter analysis: Success

Uh oh!

codecov-commenter commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Jan 9, 2026 •

edited

Loading

codecov-commenter commented Jan 9, 2026 •

edited

Loading