Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
aa0bc54
fix(memory): improve recall delivery and multilingual fallback
yiliang114 Aug 8, 2026
ec0c746
fix(memory): bound heuristic recall scoring
yiliang114 Aug 8, 2026
143a372
test(memory): pin initial recall budget with fake timers
yiliang114 Aug 8, 2026
7a079d1
test(memory): pin recall budget and scoring contracts
yiliang114 Aug 8, 2026
c9bec59
fix(memory): preserve recall field weighting
yiliang114 Aug 9, 2026
bbd040d
fix(memory): recall relevant topics beyond scan cap (#8803)
yiliang114 Aug 11, 2026
d8b4e2b
fix(memory): address recall review feedback
yiliang114 Aug 12, 2026
da48034
test(memory): measure recall rollout gate against the pre-change scorer
Aug 12, 2026
c50c69d
fix(memory): deliver a deterministic fast recall result on the initia…
Aug 12, 2026
30dda5e
docs(memory): record the fast-path decision and phase/strategy split
Aug 12, 2026
4563b19
test(memory): report the mixed-language slice in the rollout gate
Aug 12, 2026
d6aa8c5
docs(memory): align recall docs on the deterministic fast path
Aug 12, 2026
4cad8a3
Merge branch 'main' into codex/7040-memory-recall
wenshao Aug 12, 2026
823ac2a
fix(memory): use Array<T> for the fast-path test doc lists
Aug 12, 2026
5d3f9b9
docs(memory): clarify recall delivery telemetry
yiliang114 Aug 13, 2026
9adc612
fix(memory): report already-delivered recall count
yiliang114 Aug 13, 2026
ef4c5fa
Merge remote-tracking branch 'origin/main' into HEAD
yiliang114 Aug 13, 2026
6085ef0
docs(memory): align recall delivery claims
yiliang114 Aug 13, 2026
57281e8
Merge branch 'main' into codex/7040-memory-recall
yiliang114 Aug 17, 2026
8a5a524
fix(memory): rank ties by recency and record fast-delivered discards
yiliang114 Aug 18, 2026
d2598bd
docs(memory): state the candidate-cap trade and the per-turn document…
yiliang114 Aug 18, 2026
01ef7d7
fix(memory): end the initial recall wait on the fast result, widen to…
yiliang114 Aug 18, 2026
db006f8
docs(memory): correct the initial-turn preference claim, pin it with …
yiliang114 Aug 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions docs/design/2026-08-08-native-memory-recall-reliability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Native Memory Recall Reliability

## Problem

Managed-memory recall starts asynchronously for each user query. The initial
request currently performs a zero-wait consume, so a useful selector result can
miss the first prompt because of incidental scheduling. If the turn has no tool
call, that result has no later safe delivery point.

The model selector remains the normal precision gate. Its failure fallback has
two independent correctness problems: it tokenizes only ASCII text and gives
every non-empty document a positive score even without a lexical match.

## Decision

Keep the existing single-result recall lifecycle and model-primary selection.

- Give user-query recall a fixed 100 ms initial wait budget.
- Deliver a result that settles inside the budget in the initial prompt.
- If the budget expires, leave recall pending for the existing same-query
ToolResult delivery point.
- Do not abort recall or run heuristic fallback merely because the initial
budget expires.
- Preserve the existing cancellation and exactly-once terminal telemetry paths.

The 100 ms budget stays internal, per RFC #7040's direction of a small fixed
internal budget determined by benchmark rather than exposed as public
configuration; telemetry can show whether a later change is justified.

Improve only the existing failure fallback:

- normalize query and document text with Unicode NFKC;
- keep ASCII alphanumeric tokens of at least three characters;
- generate Unicode code-point bigrams for Han, Hiragana, Katakana, and Hangul
runs;
- ignore isolated CJK characters;
- bound fallback query tokens while retaining tokens from both ends;
- score only the body window that can be surfaced in the prompt;
- require a title, description, or body lexical match before applying a type
boost;
- weight title and description matches above body-only matches.

## Non-goals

- No Fast/Refined two-stage architecture.
- No public recall timing or retrieval-mode setting.
- No new tokenizer or retrieval dependency.
- No change to memory writes, scopes, extraction, DREAM, forget, or compaction.
- No removal of the shared scanner's 200-document cap. Recall-specific broad
candidate selection needs its own bounded manifest design and remains a
separate change.
Comment thread
yiliang114 marked this conversation as resolved.
Resolved

## Verification

- Recall settling inside the budget is delivered initially.
- A deadline miss leaves recall alive for later ToolResult delivery.
- Cancellation ends the bounded wait and prevents stale delivery.
- Slow recall is released by the fixed-budget timer before the initial request.
- A labeled set covers Chinese, English, Japanese, Korean, mixed text,
NFKC normalization, body-only matches, and no-result queries.
- Long CJK queries keep bounded scoring work and preserve both query ends.
- Existing active-tool noise filtering and model-selector fallback behavior
remain unchanged.
271 changes: 256 additions & 15 deletions packages/core/src/core/client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4965,9 +4965,11 @@ hello
);
});

it('should not block the main request when auto-memory recall is slow', async () => {
// Recall never settles — settledAt stays null so the UserQuery consume
// point skips it and turn.run() is called immediately without memory.
it('should hold the main request for exactly the initial recall budget when recall never settles', async () => {
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
// Recall never settles. Fake timers pin the budget contract: the
// request must still be blocked 1 ms inside the budget and proceed,
// without memory, the moment the budget expires.
vi.useFakeTimers();
mockMemoryManager.recall.mockReturnValue(new Promise(() => {}));

const mockStream = (async function* () {
Expand All @@ -4981,16 +4983,23 @@ hello
};
client['chat'] = mockChat as GeminiChat;

const stream = client.sendMessageStream(
[{ text: 'Quick question' }],
new AbortController().signal,
'prompt-id-slow-memory',
const done = fromAsync(
client.sendMessageStream(
[{ text: 'Quick question' }],
new AbortController().signal,
'prompt-id-slow-memory',
),
);
for await (const _ of stream) {
// consume stream
}

// turn.run() must have been called without the slow memory
// Drain microtasks up to the consume point, then stop 1 ms short of
// the 100 ms budget: the request must still be held.
await vi.advanceTimersByTimeAsync(99);
expect(mockTurnRunFn).not.toHaveBeenCalled();

// Budget expiry: the request proceeds without the slow memory.
await vi.advanceTimersByTimeAsync(1);
await done;

expect(mockTurnRunFn).toHaveBeenCalledWith(
'test-model',
expect.not.arrayContaining([
Expand All @@ -5000,6 +5009,61 @@ hello
);
});

it('should end the initial wait early when recall settles inside the budget', async () => {
// Fake timers pin the early-exit contract: once recall settles the
// request proceeds immediately with the memory — it must not run out
// the remaining budget. Dropping the settle listener in
// tryConsumeMemoryPrefetch would leave the request blocked until the
// full budget, which this assertion catches.
vi.useFakeTimers();
mockMemoryManager.recall.mockImplementation(
() =>
new Promise((resolve) => {
setTimeout(
() =>
resolve({
prompt: '## Relevant memory\n\nBounded memory result.',
selectedDocs: [],
strategy: 'model',
}),
10,
);
}),
);

mockTurnRunFn.mockReturnValue(
(async function* () {
yield { type: 'content', value: 'Hello' };
})(),
);
client['chat'] = {
addHistory: vi.fn(),
getHistory: vi.fn().mockReturnValue([]),
} as unknown as GeminiChat;

const done = fromAsync(
client.sendMessageStream(
[{ text: 'Quick question' }],
new AbortController().signal,
'prompt-id-bounded-memory',
),
);

// Recall settles 10 ms in; the request must already be proceeding,
// 90 ms short of the budget.
await vi.advanceTimersByTimeAsync(10);
expect(mockTurnRunFn).toHaveBeenCalled();
await done;

expect(mockTurnRunFn).toHaveBeenCalledWith(
'test-model',
expect.arrayContaining([
'## Relevant memory\n\nBounded memory result.',
]),
expect.any(AbortSignal),
);
});

it('should inject auto-memory at UserQuery consume point when recall already settled', async () => {
// mockResolvedValue settles synchronously; by the time the consume-point
// check runs (after at least one await), settledAt is set.
Expand Down Expand Up @@ -5154,11 +5218,13 @@ hello
strategy: 'model';
}) => void)
| undefined;
mockMemoryManager.recall.mockReturnValue(
new Promise((resolve) => {
let recallSignal: AbortSignal | undefined;
mockMemoryManager.recall.mockImplementation((_root, _query, options) => {
recallSignal = options.abortSignal;
return new Promise((resolve) => {
resolveRecall = resolve;
}),
);
});
});

// The model requests a tool call so pendingToolCalls is non-empty and
// the prefetch is preserved for the subsequent ToolResult turn.
Expand Down Expand Up @@ -5201,6 +5267,7 @@ hello
]),
expect.any(AbortSignal),
);
expect(recallSignal?.aborted).toBe(false);

// Recall settles between turns
resolveRecall!({
Expand Down Expand Up @@ -5349,6 +5416,180 @@ hello
expect(abortHandlerInvoked).toBe(true);
});

it('should end the bounded initial wait when the prefetch is cancelled', async () => {
vi.useFakeTimers();
const controller = new AbortController();
const handle = {
promise: new Promise<never>(() => {}),
settledAt: null,
result: null,
consumed: false,
terminalLogged: false,
firedAt: Date.now(),
controller,
};
client['pendingMemoryPrefetch'] = handle;
const privateClient = client as unknown as {
tryConsumeMemoryPrefetch: (
deliveryPoint: 'initial',
waitMs: number,
) => Promise<unknown>;
cancelPendingMemoryPrefetch: (reason: 'abort') => void;
};

const consume = privateClient.tryConsumeMemoryPrefetch('initial', 100);
setTimeout(() => privateClient.cancelPendingMemoryPrefetch('abort'), 10);
await vi.advanceTimersByTimeAsync(10);

await expect(consume).resolves.toBeNull();
expect(controller.signal.aborted).toBe(true);
expect(client['pendingMemoryPrefetch']).toBeUndefined();
});

it('should not consume a prefetch replaced during the bounded wait', async () => {
vi.useFakeTimers();
type RecallResult = {
prompt: string;
selectedDocs: Array<{
type: 'user';
filePath: string;
relativePath: string;
filename: string;
title: string;
description: string;
body: string;
mtimeMs: number;
}>;
strategy: 'model';
};
let settleRecall: ((value: RecallResult) => void) | undefined;
const handle = {
promise: new Promise<RecallResult>((resolve) => {
settleRecall = resolve;
}),
settledAt: null as number | null,
result: null,
consumed: false,
terminalLogged: false,
firedAt: Date.now(),
controller: new AbortController(),
};
client['pendingMemoryPrefetch'] = handle;
const privateClient = client as unknown as {
tryConsumeMemoryPrefetch: (
deliveryPoint: 'initial',
waitMs: number,
) => Promise<unknown>;
};

const consume = privateClient.tryConsumeMemoryPrefetch('initial', 100);

// The handle is replaced mid-wait and only settles afterwards; the
// post-wait guard must refuse the stale handle instead of consuming
// it.
const replacement = { ...handle, controller: new AbortController() };
setTimeout(() => {
client['pendingMemoryPrefetch'] = replacement;
}, 10);
setTimeout(() => {
handle.settledAt = Date.now();
settleRecall!({
prompt: '## Relevant memory\n\nReplaced result.',
selectedDocs: [],
strategy: 'model',
});
}, 20);

await vi.advanceTimersByTimeAsync(100);

await expect(consume).resolves.toBeNull();
expect(handle.consumed).toBe(false);
expect(client['pendingMemoryPrefetch']).toBe(replacement);
});

it('should not apply the initial wait budget on Cron turns', async () => {
// Cron recall fires too, but its consume point is zero-wait: with a
// never-settling recall the Cron request must proceed at elapsed 0
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
// instead of being held for the user-query budget.
vi.useFakeTimers();
mockMemoryManager.recall.mockReturnValue(new Promise(() => {}));

mockTurnRunFn.mockReturnValue(
(async function* () {
yield { type: 'content', value: 'Cron response' };
})(),
);
client['chat'] = {
addHistory: vi.fn(),
getHistory: vi.fn().mockReturnValue([]),
} as unknown as GeminiChat;

const done = fromAsync(
client.sendMessageStream(
[{ text: 'Scheduled sweep' }],
new AbortController().signal,
'prompt-id-cron-memory',
{ type: SendMessageType.Cron },
),
);

// Zero elapsed: the Cron turn must not be held by the recall budget.
await vi.advanceTimersByTimeAsync(0);
expect(mockTurnRunFn).toHaveBeenCalledWith(
'test-model',
expect.not.arrayContaining([
expect.stringContaining('Relevant memory'),
]),
expect.any(AbortSignal),
);
await done;
});

it('should keep the ToolResult consume point zero-wait', async () => {
// The ToolResult delivery point must never block on the recall
// budget: with a still-pending prefetch the ToolResult turn proceeds
// at elapsed 0 and without memory.
vi.useFakeTimers();
client['pendingMemoryPrefetch'] = {
promise: new Promise<never>(() => {}),
settledAt: null,
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
result: null,
consumed: false,
terminalLogged: false,
firedAt: Date.now(),
controller: new AbortController(),
};

mockTurnRunFn.mockReturnValue(
(async function* () {
yield { type: 'content', value: 'tool result turn' };
})(),
);
client['chat'] = {
addHistory: vi.fn(),
getHistory: vi.fn().mockReturnValue([]),
} as unknown as GeminiChat;

const done = fromAsync(
client.sendMessageStream(
[{ functionResponse: { name: 'foo', response: { ok: true } } }],
new AbortController().signal,
'prompt-id-tool-result-zero-wait',
{ type: SendMessageType.ToolResult },
),
);

await vi.advanceTimersByTimeAsync(0);
expect(mockTurnRunFn).toHaveBeenCalledWith(
'test-model',
expect.not.arrayContaining([
expect.stringContaining('Relevant memory'),
]),
expect.any(AbortSignal),
);
await done;
});

it('should abort the previous prefetch when a new UserQuery arrives mid-flight', async () => {
// Pending recall on first UserQuery — never resolves on its own.
const abortSignals: AbortSignal[] = [];
Expand Down
Loading
Loading