Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
aa0bc54
fix(memory): improve recall delivery and multilingual fallback
yiliang114 Aug 8, 2026
ec0c746
fix(memory): bound heuristic recall scoring
yiliang114 Aug 8, 2026
143a372
test(memory): pin initial recall budget with fake timers
yiliang114 Aug 8, 2026
7a079d1
test(memory): pin recall budget and scoring contracts
yiliang114 Aug 8, 2026
c9bec59
fix(memory): preserve recall field weighting
yiliang114 Aug 9, 2026
bbd040d
fix(memory): recall relevant topics beyond scan cap (#8803)
yiliang114 Aug 11, 2026
d8b4e2b
fix(memory): address recall review feedback
yiliang114 Aug 12, 2026
da48034
test(memory): measure recall rollout gate against the pre-change scorer
Aug 12, 2026
c50c69d
fix(memory): deliver a deterministic fast recall result on the initia…
Aug 12, 2026
30dda5e
docs(memory): record the fast-path decision and phase/strategy split
Aug 12, 2026
4563b19
test(memory): report the mixed-language slice in the rollout gate
Aug 12, 2026
d6aa8c5
docs(memory): align recall docs on the deterministic fast path
Aug 12, 2026
4cad8a3
Merge branch 'main' into codex/7040-memory-recall
wenshao Aug 12, 2026
823ac2a
fix(memory): use Array<T> for the fast-path test doc lists
Aug 12, 2026
5d3f9b9
docs(memory): clarify recall delivery telemetry
yiliang114 Aug 13, 2026
9adc612
fix(memory): report already-delivered recall count
yiliang114 Aug 13, 2026
ef4c5fa
Merge remote-tracking branch 'origin/main' into HEAD
yiliang114 Aug 13, 2026
6085ef0
docs(memory): align recall delivery claims
yiliang114 Aug 13, 2026
57281e8
Merge branch 'main' into codex/7040-memory-recall
yiliang114 Aug 17, 2026
8a5a524
fix(memory): rank ties by recency and record fast-delivered discards
yiliang114 Aug 18, 2026
d2598bd
docs(memory): state the candidate-cap trade and the per-turn document…
yiliang114 Aug 18, 2026
01ef7d7
fix(memory): end the initial recall wait on the fast result, widen to…
yiliang114 Aug 18, 2026
db006f8
docs(memory): correct the initial-turn preference claim, pin it with …
yiliang114 Aug 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions docs/design/2026-08-08-native-memory-recall-reliability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Native Memory Recall Reliability

## Problem

Managed-memory recall starts asynchronously for each user query. The initial
request currently performs a zero-wait consume, so a useful selector result can
miss the first prompt because of incidental scheduling. If the turn has no tool
call, that result has no later safe delivery point.

The model selector remains the normal precision gate. Its failure fallback has
two independent correctness problems: it tokenizes only ASCII text and gives
every non-empty document a positive score even without a lexical match.

## Decision

Keep the existing single-result recall lifecycle and model-primary selection.

- Give user-query recall a fixed 100 ms initial wait budget.
- Deliver a result that settles inside the budget in the initial prompt.
- If the budget expires, leave recall pending for the existing same-query
ToolResult delivery point.
- Do not abort recall or run heuristic fallback merely because the initial
budget expires.
- Preserve the existing cancellation and exactly-once terminal telemetry paths.

The 100 ms budget stays internal. It leaves margin under the RFC's 150 ms
initial-turn overhead target while telemetry can show whether a later change is
justified.

Improve only the existing failure fallback:

- normalize query and document text with Unicode NFKC;
- keep ASCII alphanumeric tokens of at least three characters;
- generate Unicode code-point bigrams for Han, Hiragana, Katakana, and Hangul
runs;
- ignore isolated CJK characters;
- bound fallback query tokens while retaining tokens from both ends;
- score only the body window that can be surfaced in the prompt;
- require a title, description, or body lexical match before applying a type
boost;
- weight title and description matches above body-only matches.

## Non-goals

- No Fast/Refined two-stage architecture.
- No public recall timing or retrieval-mode setting.
- No new tokenizer or retrieval dependency.
- No change to memory writes, scopes, extraction, DREAM, forget, or compaction.
- No removal of the shared scanner's 200-document cap. Recall-specific broad
candidate selection needs its own bounded manifest design and remains a
separate change.
Comment thread
yiliang114 marked this conversation as resolved.
Resolved

## Verification

- Recall settling inside the budget is delivered initially.
- A deadline miss leaves recall alive for later ToolResult delivery.
- Cancellation ends the bounded wait and prevents stale delivery.
- Slow recall is released by the fixed-budget timer before the initial request.
- A labeled set covers Chinese, English, Japanese, Korean, mixed text,
NFKC normalization, body-only matches, and no-result queries.
- Long CJK queries keep bounded scoring work and preserve both query ends.
- Existing active-tool noise filtering and model-selector fallback behavior
remain unchanged.
90 changes: 83 additions & 7 deletions packages/core/src/core/client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4965,9 +4965,9 @@ hello
);
});

it('should not block the main request when auto-memory recall is slow', async () => {
// Recall never settles — settledAt stays null so the UserQuery consume
// point skips it and turn.run() is called immediately without memory.
it('should bound the main request wait when auto-memory recall is slow', async () => {
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
// Recall never settles, so the UserQuery consume point proceeds after
// the fixed initial budget without memory.
mockMemoryManager.recall.mockReturnValue(new Promise(() => {}));

const mockStream = (async function* () {
Expand Down Expand Up @@ -5000,6 +5000,49 @@ hello
);
});

it('should inject auto-memory when recall settles inside the initial wait budget', async () => {
mockMemoryManager.recall.mockImplementation(
() =>
new Promise((resolve) => {
setTimeout(
() =>
resolve({
prompt: '## Relevant memory\n\nBounded memory result.',
selectedDocs: [],
strategy: 'model',
}),
10,
);
}),
);

mockTurnRunFn.mockReturnValue(
(async function* () {
yield { type: 'content', value: 'Hello' };
})(),
);
client['chat'] = {
addHistory: vi.fn(),
getHistory: vi.fn().mockReturnValue([]),
} as unknown as GeminiChat;

await fromAsync(
client.sendMessageStream(
[{ text: 'Quick question' }],
new AbortController().signal,
'prompt-id-bounded-memory',
),
);

expect(mockTurnRunFn).toHaveBeenCalledWith(
'test-model',
expect.arrayContaining([
'## Relevant memory\n\nBounded memory result.',
]),
expect.any(AbortSignal),
);
});

it('should inject auto-memory at UserQuery consume point when recall already settled', async () => {
// mockResolvedValue settles synchronously; by the time the consume-point
// check runs (after at least one await), settledAt is set.
Expand Down Expand Up @@ -5154,11 +5197,13 @@ hello
strategy: 'model';
}) => void)
| undefined;
mockMemoryManager.recall.mockReturnValue(
new Promise((resolve) => {
let recallSignal: AbortSignal | undefined;
mockMemoryManager.recall.mockImplementation((_root, _query, options) => {
recallSignal = options.abortSignal;
return new Promise((resolve) => {
resolveRecall = resolve;
}),
);
});
});

// The model requests a tool call so pendingToolCalls is non-empty and
// the prefetch is preserved for the subsequent ToolResult turn.
Expand Down Expand Up @@ -5201,6 +5246,7 @@ hello
]),
expect.any(AbortSignal),
);
expect(recallSignal?.aborted).toBe(false);

// Recall settles between turns
resolveRecall!({
Expand Down Expand Up @@ -5349,6 +5395,36 @@ hello
expect(abortHandlerInvoked).toBe(true);
});

it('should end the bounded initial wait when the prefetch is cancelled', async () => {
vi.useFakeTimers();
const controller = new AbortController();
const handle = {
promise: new Promise<never>(() => {}),
settledAt: null,
result: null,
consumed: false,
terminalLogged: false,
firedAt: Date.now(),
controller,
};
client['pendingMemoryPrefetch'] = handle;
const privateClient = client as unknown as {
tryConsumeMemoryPrefetch: (
deliveryPoint: 'initial',
waitMs: number,
) => Promise<unknown>;
cancelPendingMemoryPrefetch: (reason: 'abort') => void;
};

const consume = privateClient.tryConsumeMemoryPrefetch('initial', 100);
setTimeout(() => privateClient.cancelPendingMemoryPrefetch('abort'), 10);
await vi.advanceTimersByTimeAsync(10);

await expect(consume).resolves.toBeNull();
expect(controller.signal.aborted).toBe(true);
expect(client['pendingMemoryPrefetch']).toBeUndefined();
});

it('should abort the previous prefetch when a new UserQuery arrives mid-flight', async () => {
// Pending recall on first UserQuery — never resolves on its own.
const abortSignals: AbortSignal[] = [];
Expand Down
58 changes: 43 additions & 15 deletions packages/core/src/core/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ import { PermissionMode, type StopHookOutput } from '../hooks/types.js';

const MAX_TURNS = 100;
const MAX_RECENT_TOOL_NAMES_FOR_MEMORY = 20;
const INITIAL_MEMORY_RECALL_WAIT_MS = 100;

export enum SendMessageType {
UserQuery = 'userQuery',
Expand Down Expand Up @@ -289,9 +290,9 @@ function sameActiveGoalProjection(
* Lifecycle:
* 1. Created on UserQuery/Cron — the recall promise fires immediately,
* `pendingMemoryPrefetch` is set to this handle.
* 2. Consumed at either of two opportunistic points: a zero-wait
* `settledAt !== null` poll just before the UserQuery main request,
* or — if recall hadn't settled yet — on the first ToolResult turn.
* 2. Consumed at either of two points: a bounded wait just before the
* UserQuery main request, or — if recall remains pending — on the first
* ToolResult turn.
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
* 3. Aborted-and-discarded by every cleanup path (resetChat,
* MaxSessionTurns, etc.) or replaced when a new UserQuery arrives.
*/
Expand Down Expand Up @@ -823,18 +824,47 @@ export class GeminiClient {
}

/**
* Atomically consume the pending prefetch if it has already settled.
* Returns the recall result (caller decides where to inject it in
* `requestToSend`), or `null` if there's nothing to consume yet.
* Atomically consume the pending prefetch, optionally waiting for a bounded
* initial-turn budget. Budget expiry leaves the recall running for the next
* safe delivery point.
*
* Centralises the consume-and-mark dance so the UserQuery and ToolResult
* inject sites can't drift on the guard logic.
*/
private async tryConsumeMemoryPrefetch(
deliveryPoint: Exclude<MemoryRecallDeliveryPoint, 'discarded'>,
waitMs = 0,
): Promise<RelevantAutoMemoryPromptResult | null> {
const handle = this.pendingMemoryPrefetch;
if (!handle || handle.settledAt === null || handle.consumed) {
if (!handle || handle.consumed) {
return null;
}

if (handle.settledAt === null && waitMs > 0) {
await new Promise<void>((resolve) => {
const finish = () => {
clearTimeout(timer);
handle.controller.signal.removeEventListener('abort', finish);
resolve();
};

const timer = setTimeout(finish, waitMs);
if (handle.controller.signal.aborted) {
finish();
} else {
handle.controller.signal.addEventListener('abort', finish, {
once: true,
});
void handle.promise.then(finish, finish);
}
});
}

if (
this.pendingMemoryPrefetch !== handle ||
handle.settledAt === null ||
handle.consumed
) {
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
return null;
}
handle.consumed = true;
Expand Down Expand Up @@ -2951,14 +2981,12 @@ export class GeminiClient {
}
}

// Zero-wait poll: consume only if the prefetch has already settled.
// Done AFTER the async reminder setup above so recall settling during
// those awaits still gets caught here. (settledAt is set in
// promise.finally(); microtask ordering guarantees it's visible
// after any await prior to this point — flatMapTextParts above is
// the natural drain.) If still not settled, skip — the ToolResult
// inject point will retry on the next turn.
const userQueryMemory = await this.tryConsumeMemoryPrefetch('initial');
const userQueryMemory = await this.tryConsumeMemoryPrefetch(
'initial',
messageType === SendMessageType.UserQuery
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
? INITIAL_MEMORY_RECALL_WAIT_MS
: 0,
);
if (userQueryMemory?.prompt) {
// Unshift to the front of systemReminders: on a UserQuery turn
// requestToSend leads with user text, so positioning memory at
Expand Down
Loading
Loading