Skip to content

feat(context-offloader): let a tool call opt out of offloading via invocation_state - #4327

Closed
strandly-the-agent wants to merge 4 commits into
strands-agents:mainfrom
strandly-the-agent:strandly/context-offloader-skip-key
Closed

strandly-the-agent wants to merge 4 commits into
strands-agents:mainfrom
strandly-the-agent:strandly/context-offloader-skip-key

Conversation

@strandly-the-agent

@strandly-the-agent strandly-the-agent commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Description

ContextOffloader hooks AfterToolCallEvent, so it offloads every oversized tool result — including results that are never meant for the model. When a tool invokes other tools through the executor and consumes their output in code (a programmatic/direct tool-calling pattern), that code receives a [Offloaded: …] preview instead of the data: json.loads on a preview fails, file contents arrive truncated, and the failure is silent.

This adds SKIP_CONTEXT_OFFLOAD_KEY ("strands:skip_context_offload"), exported from strands.vended_plugins.context_offloader. When it is truthy in a tool call's invocation_state, _handle_tool_result returns early and the result passes through untouched. The key rides on invocation_state because that is what ToolExecutor._stream already threads to the hook, so a caller can opt out without a new hook or a plugin option; the delegate-tool skip a few lines above is the same shape.

The key applies to every tool call sharing that invocation_state (the agent loop passes one dict to all tool calls of an invocation), so the docstring tells callers to pass a copy to scope it to one call. A direct call takes it as a kwarg:

from strands.vended_plugins.context_offloader import SKIP_CONTEXT_OFFLOAD_KEY

result = agent.tool.fetch_report(**{SKIP_CONTEXT_OFFLOAD_KEY: True}, record_direct_tool_call=False)

Related Issues

None. TypeScript parity is in a separate PR (same key, camelCased to strands:skipContextOffload per the strands:model_routing / strands:modelRouting precedent).

Documentation PR

Included: a short Opting a call out section in site/…/plugins/context-offloader.mdx.

Type of Change

New feature

Testing

  • ruff check / ruff format --check clean on the touched files; pytest tests/strands/vended_plugins/context_offloader/test_plugin.py → 91 passed (2 new: key set → result and storage untouched; key falsy → offloaded as before).

  • mypy on the package reports the same two pre-existing @hook overload errors with and without the change; complexipy for _handle_tool_result goes 57 → 58 (already over threshold before this PR — the guard is one extra early return).

  • Exercised end to end: a real Agent with ContextOffloader(max_result_tokens=50) and a tool that calls another tool via ToolExecutor._stream with {**invocation_state, SKIP_CONTEXT_OFFLOAD_KEY: True} — the inner code received the full 4,000-char payload, and the outer tool's own result was still offloaded.

  • One independent review pass; its findings (state the invocation-wide scope, show a public route, whole-object test assertions) are folded in.

  • I ran the relevant checks (hatch run prepare equivalents run individually: ruff, mypy, pytest)

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Opened by strandly-the-agent at the request of mkmeral.

Tools that are invoked directly by other code (not by the model) need the
full result, not a preview. A truthy SKIP_OFFLOAD_KEY in the call's
invocation_state exempts that result from offloading.
@strandly-the-agent
strandly-the-agent requested a review from a team as a code owner September 14, 2026 11:25
@github-actions github-actions Bot added area-interventions Related to interventions area-hooks Features or requests that might be implementable via hooks enhancement New feature or request python Pull requests that update python code labels Sep 14, 2026
@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Closing per lizradway's note on #4328: ContextOffloader is being deprecated in favour of the context manager, and the rewired context_manager="auto" (#4282) truncates on MessageAddedEvent, so tool calls that never enter the message history (the programmatic/direct-call case this was for) are already left alone. Verified on main: an inner call through ToolExecutor._stream with context_manager="auto" and no explicit offloader returns the full payload. The fix belongs on the consumer side (stop adding an explicit ContextOffloader once on that release), not in the SDK.

This branch is waiting to be deployed

1 waiting deployment
manual-approval — 9e7e8184 Waiting Sep 14, 2026 by strandly-the-agent via Build and Deploy Preview #1736
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-hooks Features or requests that might be implementable via hooks area-interventions Related to interventions enhancement New feature or request python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants