fix(hindsight): make hindsight_retain tool fire-and-forget - #22836
Draft
bitfieldz wants to merge 1 commit into
Draft
fix(hindsight): make hindsight_retain tool fire-and-forget#22836bitfieldz wants to merge 1 commit into
bitfieldz wants to merge 1 commit into
Conversation
The user-facing hindsight_retain tool was calling client.aretain()
which blocks server-side until the embedding/extraction LLM completes.
When the inference backend is under load, extraction routinely exceeds
HINDSIGHT_TIMEOUT (default 120s), the SDK raises a bare timeout, and
the tool returns '{"error": "Failed to store memory: "}' (note empty
inner exception) to the model.
This is asymmetric with the auto-retain (sync_turn -> _do_retain) path,
which already uses aretain_batch(retain_async=True) for instant return
after server-side queuing. The user-callable tool just never got the
same treatment. The SDK's aretain() is a thin wrapper over aretain_batch
but does not expose retain_async, so it's hardcoded blocking.
Switch the tool path to aretain_batch(retain_async=True). Server-side
extraction still happens asynchronously and is observable via
/v1/default/banks/<bank>/operations; recall surfaces the extracted
memory units once Reflect runs. No client-side timeout coupling to
LLM load.
The success message changes from 'Memory stored successfully.' to
'Memory queued for storage.' to make the fire-and-forget semantics
explicit to the model.
This was referenced Jul 8, 2026
Contributor
|
Thanks for identifying the missing call-level Problems
Suggested changes
Automated hermes-sweeper review. |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
hindsight_retainwas callingclient.aretain(), a thin SDK wrapper that hardcodesretain_async=False. The tool blocks until the server finishes LLM extraction. Under load,extraction exceeds
_DEFAULT_TIMEOUT(120 s); the SDK raises a bareTimeoutError; the toolreturns
{"error": "Failed to store memory: "}with an empty exception string._do_retain(the auto-retain path) has usedaretain_batch(retain_async=True)sincea1b6d9e — the tool just never
got the same treatment. This PR brings them in line: route
hindsight_retainthrougharetain_batch(retain_async=True), eliminating the timeout coupling entirely. Extraction runsserver-side; results surface via
recallonce Reflect runs.Success message changes from
"Memory stored successfully."to"Memory queued for storage."to reflect the queued semantics.
Related Issue
Partially addresses #14950 — resolves the
retaincase by eliminating the timeout dependency.The
reflecttimeout described in that issue is out of scope here.Type of Change
Changes Made
plugins/memory/hindsight/__init__.py: replaceclient.aretain(**retain_kwargs)withclient.aretain_batch(bank_id=..., items=[retain_kwargs], retain_async=True)inhandle_tool_call("hindsight_retain"). Update success message.tests/plugins/memory/test_hindsight_provider.py: update fiveTestToolHandlersretain teststo assert
aretain_batch.call_args.kwargsinstead ofaretain.call_args.kwargs.How to Test
pytest tests/plugins/memory/test_hindsight_provider.py -q— 96 passed.pytest tests/plugins/ -q— 539 passed. (One pre-existing flake:test_achievements_plugin.py::test_evaluate_all_stale_cache_serves_stale_and_refreshes_in_background— timing race, reproducible on clean
main, unrelated.)hindsight_retainreturns{"result": "Memory queued for storage."}immediately; the operation appears atGET /v1/default/banks/<bank>/operationswithin ~200 ms.Checklist
Code
pytest tests/plugins/ -q— 539 passedDocumentation & Housekeeping
cli-config.yaml.example— N/ACONTRIBUTING.md/AGENTS.md— N/Acontent,context,tags) unchanged; success string updatedScreenshots / Logs
Before:
After: