fix(memory-tools): #215-class — auth_headers on commit_memory + search_memory HTTP fallback - #285
Merged
Merged
Conversation
Contributor
Author
|
PM review (cannot self-approve — needs second reviewer): Diff is correct. Both This also explains |
…h_memory HTTP fallback Context: platform now gates `GET /workspaces/:id/memories` and `POST /workspaces/:id/memories` behind workspace auth (post-#166 / #167 AdminAuth wave). The `builtin_tools.memory` tool had three HTTP call sites: 1. commit_memory POST fallback (line 121) ← NO auth_headers 2. search_memory GET fallback (line 269) ← NO auth_headers 3. activity-log helper POST (line 371) ← HAS auth_headers Path 3 was already fixed. Paths 1 + 2 silently 401 every call, but the tool's error-handling path returns `{"success": False}` without surfacing the auth failure to the agent. Result: the agent sees an empty memory backlog on every call and assumes there's nothing to do. ## Discovered today Technical Researcher is the first workspace opted in to the idle-loop pilot from #216 (reflection-on-completion pattern). The pilot fires every 10 min, the agent calls `search_memory "research-backlog:..."` as the first step, gets back an empty result, writes "tr-idle clean" to memory, and stops. Clean-idle outcome every tick, 9 consecutive ticks. Looking at TR's activity_logs response bodies: "Memory auth has failed on every tick this session — skipping the call" "tr-idle — step 2 done. Memory unavailable (auth token missing..." "tr-idle 04:15 — clean (memory auth still down, 3rd consecutive tick)" The AGENT knew the memory calls were failing. The platform 401 error was surfacing in the tool response, but our instrumentation wasn't counting it as a defect — we saw "tr-idle clean" writes and assumed the pilot was working as designed. It was actually silently broken. ## Fix Import `platform_auth.auth_headers` lazily (same pattern as the activity-log path already uses), attach `headers=_auth()` to both httpx call sites. Matches the #225 fix for the register call. ## Not in this PR - awareness_client.py also makes HTTP calls to a separate AWARENESS_URL service (not the platform), which may or may not need the same fix depending on that service's auth posture. Out of scope for this PR. - TR's specific token problem: TR's `/configs/.auth_token` file is empty because it was re-provisioned via `apply_template: true` (recovery path from the failed-volume incident) and Phase 30.1 only mints a token on FIRST register per workspace. This fix doesn't help TR until TR gets a fresh token — tracked separately. ## Test plan - [x] Python syntax check on memory.py passes - [ ] CI: all memory-related tests should still pass (the new code paths only add header passing, no shape change) - [ ] Real-world verification: after TR gets a fresh token, idle-loop pilot should produce a dispatch within 10 min (seeded backlog already in place from this session) ## Related - #215 / #225 — register call auth_headers fix (same pattern) - #216 — TR idle-loop pilot (couldn't measure until this lands) - #166 / #167 — platform AdminAuth wave that surfaced this gap
…rwrite The #215-class fix in memory.py (859a60e) adds headers=_headers to the direct-httpx commit_memory + search_memory paths, but 9 existing tests in test_memory.py had FakeAsyncClient.post/get signatures like `async def post(self, url, json):` with no headers kwarg. Python raised TypeError: unexpected keyword argument 'headers' on every call, commit_memory caught it and returned {success: False}, tests failed. Fixes applied: 1. Add `headers=None` to every FakeAsyncClient.post + .get signature across test_memory.py. Uses replace_all so all 9+ fakes match. 2. For tests that capture a single captured["url"]: - test_commit_memory_uses_awareness_client_when_configured - test_commit_memory_uses_platform_fallback_without_awareness - test_commit_memory_httpx_201_success filter to only capture /memories URLs. Without the filter, the subsequent _record_memory_activity fire-and-forget post to /activity overwrites captured["url"] and the assertion fails. 3. For test_commit_memory_promoted_packet_logs_skill_promotion: bump expected captured["calls"] from 3 to 4. Pre-fix, the memory_write /activity call (from _record_memory_activity #125) was silently dropped because the fake rejected headers=; post-fix it succeeds and lands in the captured list alongside the skill_promotion /activity and /registry/heartbeat calls. Also extend that test's fake to accept /registry/heartbeat (was raising AssertionError). Total: 36/36 memory tests pass. Full workspace-template suite 1189/1189. This is strictly test-infrastructure work — zero production code changed. CI never caught the break because the Mac mini runner has been stuck for ~4 hours (tick-33/34/35/36 reports). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HongmingWang-Rabbit
force-pushed
the
fix/memory-tools-auth-headers
branch
from
April 16, 2026 00:29
858a60e to
469d24c
Compare
HongmingWang-Rabbit
added a commit
that referenced
this pull request
Apr 16, 2026
Companion to #285 which fixed builtin_tools/memory.py. The MCP server (a2a_mcp_server.py) uses a DIFFERENT implementation in a2a_tools.py for the same memory endpoints — tool_recall_memory() and tool_commit_memory(). These had the same missing-auth bug: httpx calls to /workspaces/:id/memories without auth_headers(), which silently 401 against the platform's workspace auth gate. #285 fixed memory.py but left a2a_tools.py untouched because I didn't realize there were two separate implementations of the same functionality. The MCP server (used by Claude Code runtime) calls a2a_tools.py; the LangGraph adapter calls builtin_tools/memory.py. Only the LangGraph path was fixed. Root cause confirmation: `auth_headers()` works inside TR's container (returns a valid 43-char Bearer), and `search_memory` via Python import returns 26 results including the seeded backlog. But the Claude Code agent says "Memory auth still broken" because it calls `recall_memory` via MCP → a2a_tools.py → unauthenticated httpx → 401. Fix: add `headers=_auth_headers_for_heartbeat()` to both httpx calls. The helper `_auth_headers_for_heartbeat()` already existed at line 23 (used for the heartbeat POST at line 68) — it lazily imports platform_auth and tolerates it being absent. Identical 1-line addition per function. Closes the last blocker for the TR idle-loop pilot end-to-end signal.
HongmingWang-Rabbit
added a commit
that referenced
this pull request
Apr 16, 2026
…lligence Rollout wave 2. Technical Researcher (wave 1, #216) has been the lone pilot since 2026-04-15. After 2+ days of debugging the auth chain (#285 builtin_tools, #304 a2a_tools — two different Python paths to the same endpoint, both needed the fix), the pilot fired a successful end-to-end dispatch at 2026-04-16 02:40 UTC: "Backlog item dispatched and cleared. Done in one A2A send. Step 1: Memory search found one..." Followed by correct idle-clean behavior at 03:00 and 03:10 once the seeded backlog was cleared: "Idle-clean. Backlog is empty, no new follow-ups to file." Pilot is validated. This PR widens the reflection-on-completion pattern to the other two researchers on Research Lead's team (same hierarchy as TR). ## Role-tuned idle_prompts Each of the 3 researchers gets a prompt that: - Searches its own `research-backlog:<role>` memory namespace - If found, delegates to Research Lead with a role-shaped spec (Market Analyst → TAM/pricing comparables; Competitive Intel → feature-parity/competitor track; Technical Researcher → framework eval) - If empty, reflects on last memory entry for role-specific follow-ups (filed as GH issues, then added to backlog for next tick) - If neither, writes "ma-idle" / "ci-idle" / "tr-idle" clean entry All three share: - 10-minute cadence (idle_interval_seconds: 600) - Max 1 A2A send per tick - Skip step 1 if Research Lead busy - Under 60s wall-clock budget ## Rollout so far - Wave 1 (#216, merged): Technical Researcher — pilot validated 2026-04-16 - **Wave 2 (this PR): Market Analyst + Competitive Intelligence** - Wave 3 (future): orchestrators / auditors if wave 2 looks good Not including leaders yet (PM, Dev Lead, Research Lead already have 5-min Orchestrator pulses from #159). Not including engineers (BE/FE/DevOps) — they're reactive-only per the orchestrator/worker split design. ## Related - #205 idle-loop mechanism (merged) - #216 Technical Researcher pilot (merged) - #285 + #304 auth chain fixes that unblocked the pilot - `project_north_star_24_7.md` memory — this is the "team runs 24/7" tick
molecule-ai Bot
pushed a commit
that referenced
this pull request
Apr 21, 2026
fix(memory-tools): #215-class — auth_headers on commit_memory + search_memory HTTP fallback
2 tasks
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The platform gates `GET/POST /workspaces/:id/memories` behind workspace auth (post-#166 / #167 AdminAuth wave). `builtin_tools/memory.py` had three HTTP call sites:
The fallback paths silently 401 on every call.
Real-world impact (discovered this session)
Technical Researcher is the first workspace on the #216 idle-loop pilot (reflection-on-completion pattern from the Hermes/Letta research). The pilot fires every 10 min, agent calls `search_memory "research-backlog:..."`, gets empty result, writes "tr-idle clean", stops. Clean-idle 9 consecutive ticks.
Looking at TR's `activity_logs.response_body`:
```
"Memory auth has failed on every tick this session — skipping the call"
"tr-idle — step 2 done. Memory unavailable (auth token missing..."
"tr-idle 04:15 — clean (memory auth still down, 3rd consecutive tick)"
```
The agent was correctly diagnosing the failure. We just weren't looking at the response bodies — we saw "tr-idle clean" writes and assumed the pilot was working as designed. It was silently broken.
Fix
Import `platform_auth.auth_headers` lazily (same pattern as line 371 already uses), attach `headers=_auth()` to both httpx call sites. Mirrors the #225 fix for the register call.
Test plan
Not in this PR
Related