Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/.size-baseline
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
3480 audio-capture-prebuilds.yml
9023 auto-minimize-spam.yml
4638 build-and-publish-image.yml
29715 cd-cua-driver.yml
42519 cd-cua-driver.yml
2076 cd-mobile-mcp.yml
69782 ci.yml
1482 codeql.yml
Expand Down
17 changes: 12 additions & 5 deletions docs/design/2026-07-22-lazy-google-genai-loading.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,21 @@ Provider implementations continue to use the official SDK classes. In particular

`createContentGenerator()` still validates configuration, preloads the runtime fetch implementation, and performs Qwen OAuth credential acquisition at its current point in the session lifecycle. It returns a private lazy `ContentGenerator` whose memoized loader constructs the selected provider and wraps it in `LoggingContentGenerator` on the first asynchronous content-generator operation.

All four asynchronous operations share the same loader promise:
All three asynchronous operations share the same loader promise:

- `generateContent`
- `generateContentStream`
- `countTokens`
- `embedContent`

Concurrent first calls therefore import and construct the provider once. `useSummarizedThinking()` remains synchronous and is supplied from the selected provider's known behavior: true for Gemini/Vertex and false for OpenAI, Qwen OAuth, and Anthropic.
Concurrent first calls therefore import and construct the provider once.

> **Update (2026-08, PR #9676)**: `countTokens` and `useSummarizedThinking`
> were removed from the `ContentGenerator` interface — no production caller
> used either, and the removal narrows every provider and test double to the
Comment thread
DragonnZhang marked this conversation as resolved.
Resolved
> operations something actually calls. This doc originally listed four shared
> asynchronous operations (including `countTokens`) and a synchronous
> `useSummarizedThinking()` supplied from each provider's known behavior; both
> are gone from the interface, the lazy wrapper, and the four providers.

Qwen OAuth credential acquisition remains eager within `createContentGenerator()`. An expired or missing cached credential therefore continues to reject ACP session creation rather than producing an apparently usable session that fails only on its first prompt.

Expand All @@ -49,7 +56,7 @@ The serve fast-path metafile guard adds `@google/genai` to the ACP forbidden-pac

There are three direct production creation paths. `Config.refreshAuth()` owns the main-session generator. `BaseLlmClient` owns cached per-model generators for routed side requests. `createRuntimeContentGeneratorView()` owns dedicated generators used by the in-process agent backend, subagent manager, and forked agents. Each path stores and consumes only the `ContentGenerator` interface, so the private lazy wrapper preserves its ownership and routing boundary.

The interface consumers call only `generateContent`, `generateContentStream`, `countTokens`, `embedContent`, and `useSummarizedThinking`. The main chat path, prompt hooks, memory/goal/side queries, vision routing, subagents, and session resume do not inspect the concrete provider or unwrap `LoggingContentGenerator`; a repository-wide search found no production `instanceof` or `getWrapped()` caller. MCP tool discovery is separate from generator ownership and keeps the SDK-provided `mcpToTool` adapter behind its own first-use import.
The interface consumers call only `generateContent`, `generateContentStream`, and `embedContent`. The main chat path, prompt hooks, memory/goal/side queries, vision routing, subagents, and session resume do not inspect the concrete provider or unwrap `LoggingContentGenerator`; a repository-wide search found no production `instanceof` or `getWrapped()` caller. MCP tool discovery is separate from generator ownership and keeps the SDK-provided `mcpToTool` adapter behind its own first-use import.

## Alternatives rejected

Expand All @@ -71,7 +78,7 @@ The interface consumers call only `generateContent`, `generateContentStream`, `c

## Verification

Unit tests cover helper parity, deferred construction, Qwen credential timing, single-flight behavior, provider-specific summarized-thinking values, deferred module failures, and MCP discovery behavior. The bundled metafile must show `@google/genai` absent from the ACP static closure while retaining it in dynamic provider/MCP chunks.
Unit tests cover helper parity, deferred construction, Qwen credential timing, single-flight behavior, deferred module failures, and MCP discovery behavior. The bundled metafile must show `@google/genai` absent from the ACP static closure while retaining it in dynamic provider/MCP chunks.

The 2C4G acceptance run follows #7264: 30 paired serial cold starts, `channel.initialize` P50/P95, process-to-first-session, preheated/warm behavior, concurrent first sessions, telemetry on/off, and peak RSS. Because this change moves work later, it additionally records session-response-to-first-token and process-to-first-token for an immediate first prompt. A startup win that is fully repaid as a first-token regression is reported rather than treated as a successful optimization.

Expand Down
11 changes: 8 additions & 3 deletions docs/design/web-shell/web-shell-image-drag-and-drop.md
Original file line number Diff line number Diff line change
Expand Up @@ -502,9 +502,14 @@ oversized placeholder。它们验证既有服务端契约;Web Shell helper 测
候选顺序和 encoded-data 剩余预算。

BMP 以 `image/bmp` 进入缩略图 data URL 和 daemon image block。Core 的
`SUPPORTED_IMAGE_MIME_TYPES` 明确包含 `image/bmp`,`ImageTokenizer` 解析 BMP 尺寸,
OpenAI converter 把启用 image modality 的 `inlineData` 原样构造成
`data:image/bmp;base64,...`;Gemini 路径保留相同 `inlineData`。因此 V1 不在浏览器转码。
`SUPPORTED_IMAGE_MIME_TYPES` 明确包含 `image/bmp`,OpenAI converter 把启用 image modality 的
`inlineData` 原样构造成 `data:image/bmp;base64,...`;Gemini 路径保留相同 `inlineData`。
因此 V1 不在浏览器转码。

> **2026-08-24 同步注记(PR #9676)**:request-tokenizer 估计器簇(含 `ImageTokenizer`
Comment thread
DragonnZhang marked this conversation as resolved.
Resolved
> 及其 BMP 尺寸解析)已作为孤儿代码删除。BMP 支持现在仅依赖 `SUPPORTED_IMAGE_MIME_TYPES`
> 接受清单与 converter 透传;token 计数使用 `compactionInputSlimming.ts` 中的固定
> `DEFAULT_IMAGE_TOKEN_ESTIMATE`。下文对 BMP 路径的 E2E/人工验收要求不变。
浏览器若不能解码缩略图,不影响附件数据传输,但 E2E 必须覆盖 Chromium 解码,
Firefox/Linux 必须完成人工验收。

Expand Down
5 changes: 2 additions & 3 deletions packages/cli/src/commands/review/lib/assets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,8 @@ export const ASSET_HEADER_BYTES = 16;
*
* A sibling signature table lives in core: `sniffFileKind` in
* `packages/core/src/utils/binary-content.ts` (best-effort kind detection for
* fetched web content, deliberately looser) and the dimension extractors in
* `packages/core/src/utils/request-tokenizer/imageTokenizer.ts`. Admitting or
* correcting a format here means checking those sites too.
* fetched web content, deliberately looser). Admitting or correcting a format
* here means checking that site too.
Comment thread
DragonnZhang marked this conversation as resolved.
Resolved
*/
export function sniffImageFormat(header: Uint8Array): ImageFormat | null {
const at = (i: number): number => header[i] ?? -1;
Expand Down
6 changes: 3 additions & 3 deletions packages/cli/src/ui/AppContainer.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5080,9 +5080,9 @@ describe('AppContainer State Management', () => {
lastCompletedUuid: 'a1',
};

vi.spyOn(mockConfig, 'getContentGenerator').mockReturnValue({
useSummarizedThinking: vi.fn(() => false),
} as unknown as ReturnType<typeof mockConfig.getContentGenerator>);
vi.spyOn(mockConfig, 'getContentGenerator').mockReturnValue(
{} as unknown as ReturnType<typeof mockConfig.getContentGenerator>,
);
vi.spyOn(mockConfig, 'initialize').mockResolvedValue(undefined);
vi.spyOn(mockConfig, 'getResumedSessionData').mockReturnValue(
resumeSessionData as ReturnType<
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -701,7 +701,6 @@ describe('SessionPicker', () => {
getIdeMode: () => false,
isTrustedFolder: () => false,
getToolRegistry: () => ({ getTool: () => undefined }),
getContentGenerator: () => ({ useSummarizedThinking: () => false }),
} as unknown as Config;
const PREVIEW_SETTINGS_STUB = {
merged: { ui: {} },
Expand Down
1 change: 0 additions & 1 deletion packages/cli/src/ui/components/StandaloneSessionPicker.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ const PREVIEW_CONFIG_STUB = {
getIdeMode: () => false,
isTrustedFolder: () => false,
getToolRegistry: () => ({ getTool: () => undefined }),
getContentGenerator: () => ({ useSummarizedThinking: () => false }),
} as unknown as Config;

const PREVIEW_SETTINGS_STUB = {
Expand Down
2 changes: 0 additions & 2 deletions packages/core/src/agents/runtime/agent-headless.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,7 @@ vi.mock('../../core/contentGenerator.js', async (importOriginal) => {
createContentGenerator: vi.fn().mockResolvedValue({
generateContent: vi.fn(),
generateContentStream: vi.fn(),
countTokens: vi.fn().mockResolvedValue({ totalTokens: 100 }),
embedContent: vi.fn(),
useSummarizedThinking: vi.fn().mockReturnValue(false),
}),
createContentGeneratorConfig: vi.fn().mockReturnValue({
model: DEFAULT_QWEN_MODEL,
Expand Down
4 changes: 0 additions & 4 deletions packages/core/src/config/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5181,7 +5181,6 @@ describe('Server Config (config.ts)', () => {
vi.mocked(createContentGenerator).mockResolvedValue({
generateContent: vi.fn(),
generateContentStream: vi.fn(),
countTokens: vi.fn(),
embedContent: vi.fn(),
} as unknown as ContentGenerator);

Expand Down Expand Up @@ -5232,7 +5231,6 @@ describe('Server Config (config.ts)', () => {
vi.mocked(createContentGenerator).mockResolvedValue({
generateContent: vi.fn(),
generateContentStream: vi.fn(),
countTokens: vi.fn(),
embedContent: vi.fn(),
} as unknown as ContentGenerator);

Expand Down Expand Up @@ -5266,7 +5264,6 @@ describe('Server Config (config.ts)', () => {
vi.mocked(createContentGenerator).mockResolvedValue({
generateContent: vi.fn(),
generateContentStream: vi.fn(),
countTokens: vi.fn(),
embedContent: vi.fn(),
} as unknown as ContentGenerator);

Expand Down Expand Up @@ -5823,7 +5820,6 @@ describe('Server Config (config.ts)', () => {
vi.mocked(createContentGenerator).mockResolvedValue({
generateContent: vi.fn(),
generateContentStream: vi.fn(),
countTokens: vi.fn(),
embedContent: vi.fn(),
} as unknown as ContentGenerator);

Expand Down
19 changes: 0 additions & 19 deletions packages/core/src/core/__tests__/openaiTimeoutHandling.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -363,24 +363,5 @@ describe('OpenAIContentGenerator Timeout Handling', () => {
generator.generateContent(request, 'test-prompt-id'),
).rejects.toThrow(/Request timeout after \d+s/);
});

it('should fall back to character-based estimation if countTokens fails', async () => {
const timeoutError = new Error('Request timeout');
mockOpenAIClient.chat.completions.create.mockRejectedValue(timeoutError);

// Mock countTokens to throw error
const mockCountTokens = vi.spyOn(generator, 'countTokens');
mockCountTokens.mockRejectedValue(new Error('Count tokens failed'));

const request = {
contents: [{ role: 'user' as const, parts: [{ text: 'Hello world' }] }],
model: 'gpt-4',
};

// Should not throw due to token counting failure
await expect(
generator.generateContent(request, 'test-prompt-id'),
).rejects.toThrow(/Request timeout after \d+s/);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,19 @@

import { getEventListeners } from 'node:events';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type {
CountTokensParameters,
GenerateContentParameters,
} from '@google/genai';
import type { GenerateContentParameters } from '@google/genai';
import { FinishReason, GenerateContentResponse } from '@google/genai';
import type { ContentGeneratorConfig } from '../contentGenerator.js';
import {
DEFAULT_TIMEOUT,
DISABLED_REQUEST_TIMEOUT_MS,
} from '../openaiContentGenerator/constants.js';

// Mock the request tokenizer module BEFORE importing the class that uses it.
const mockTokenizer = {
calculateTokens: vi.fn(),
dispose: vi.fn(),
};
const mockReportAnthropicRequest = vi.hoisted(() => vi.fn());
const mockReportAnthropicFollowingRequest = vi.hoisted(() => vi.fn());
const mockReportAnthropicResponse = vi.hoisted(() => vi.fn());
const mockReportAnthropicEvent = vi.hoisted(() => vi.fn());

vi.mock('../../utils/request-tokenizer/index.js', () => ({
RequestTokenEstimator: vi.fn(() => mockTokenizer),
}));
vi.mock('../../telemetry/gen-ai-request.js', () => ({
reportAnthropicRequest: mockReportAnthropicRequest,
reportAnthropicFollowingRequest: mockReportAnthropicFollowingRequest,
Expand Down Expand Up @@ -107,16 +96,6 @@ describe('AnthropicContentGenerator', () => {
savedMaxOutputTokensEnv = process.env[MAX_OUTPUT_TOKENS_ENV];
delete process.env[MAX_OUTPUT_TOKENS_ENV];

mockTokenizer.calculateTokens.mockResolvedValue({
totalTokens: 50,
breakdown: {
textTokens: 50,
imageTokens: 0,
audioTokens: 0,
otherTokens: 0,
},
processingTime: 1,
});
anthropicState = anthropicMockState;

anthropicState.createImpl.mockReset();
Expand Down Expand Up @@ -3305,58 +3284,6 @@ describe('AnthropicContentGenerator', () => {
});
});

describe('countTokens', () => {
it('counts tokens using the request tokenizer', async () => {
const { AnthropicContentGenerator } = await importGenerator();
const generator = new AnthropicContentGenerator(
{
model: 'claude-test',
apiKey: 'test-key',
timeout: 10_000,
maxRetries: 2,
samplingParams: {},
schemaCompliance: 'auto',
},
mockConfig,
);

const request: CountTokensParameters = {
contents: [{ role: 'user', parts: [{ text: 'Hello world' }] }],
model: 'claude-test',
};

const result = await generator.countTokens(request);
expect(mockTokenizer.calculateTokens).toHaveBeenCalledWith(request);
expect(result.totalTokens).toBe(50);
});

it('falls back to character approximation when tokenizer throws', async () => {
const { AnthropicContentGenerator } = await importGenerator();
mockTokenizer.calculateTokens.mockRejectedValueOnce(new Error('boom'));
const generator = new AnthropicContentGenerator(
{
model: 'claude-test',
apiKey: 'test-key',
timeout: 10_000,
maxRetries: 2,
samplingParams: {},
schemaCompliance: 'auto',
},
mockConfig,
);

const request: CountTokensParameters = {
contents: [{ role: 'user', parts: [{ text: 'Hello' }] }],
model: 'claude-test',
};

const content = JSON.stringify(request.contents);
const expected = Math.ceil(content.length / 4);
const result = await generator.countTokens(request);
expect(result.totalTokens).toBe(expected);
});
});

describe('generateContentStream', () => {
it('emits tool preparation metadata before the complete function call', async () => {
const { AnthropicContentGenerator } = await importGenerator();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@

import Anthropic from '@anthropic-ai/sdk';
import type {
CountTokensParameters,
CountTokensResponse,
EmbedContentParameters,
EmbedContentResponse,
GenerateContentParameters,
Expand All @@ -29,7 +27,6 @@ type MessageCreateParamsNonStreaming =
Anthropic.MessageCreateParamsNonStreaming;
type MessageCreateParamsStreaming = Anthropic.MessageCreateParamsStreaming;
type RawMessageStreamEvent = Anthropic.RawMessageStreamEvent;
import { RequestTokenEstimator } from '../../utils/request-tokenizer/index.js';
import { safeJsonParse } from '../../utils/safeJsonParse.js';
import { AnthropicContentConverter } from './converter.js';
Comment thread
DragonnZhang marked this conversation as resolved.
Resolved
import { buildAnthropicUsageMetadata } from './usage.js';
Expand Down Expand Up @@ -445,41 +442,11 @@ export class AnthropicContentGenerator implements ContentGenerator {
return drainThenCleanup();
}

async countTokens(
request: CountTokensParameters,
): Promise<CountTokensResponse> {
try {
const estimator = new RequestTokenEstimator();
const result = await estimator.calculateTokens(request);

return {
totalTokens: result.totalTokens,
};
} catch (error) {
debugLogger.warn(
'Failed to calculate tokens with tokenizer, ' +
'falling back to simple method:',
error,
);

const content = JSON.stringify(request.contents);
const totalTokens = Math.ceil(content.length / 4);
return {
totalTokens,
};
}
}

async embedContent(
_request: EmbedContentParameters,
): Promise<EmbedContentResponse> {
throw new Error('Anthropic does not support embeddings.');
}

useSummarizedThinking(): boolean {
return false;
}

private buildHeaders(useProxyIdentity: boolean): Record<string, string> {
// Beta headers are computed per-request in buildPerRequestHeaders so they
// stay in sync with what the request body actually carries — see #3788
Expand Down
21 changes: 0 additions & 21 deletions packages/core/src/core/client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -424,26 +424,6 @@ vi.mock(
);
import { microcompactHistory } from '../services/microcompaction/microcompact.js';

// Mock RequestTokenizer to use simple character-based estimation
vi.mock('../utils/request-tokenizer/requestTokenizer.js', () => ({
RequestTokenizer: class {
async calculateTokens(request: { contents: unknown }) {
// Simple estimation: count characters in JSON and divide by 4
const totalChars = JSON.stringify(request.contents).length;
return {
totalTokens: Math.floor(totalChars / 4),
breakdown: {
textTokens: Math.floor(totalChars / 4),
imageTokens: 0,
audioTokens: 0,
otherTokens: 0,
},
processingTime: 0,
};
}
},
}));

/**
* Array.fromAsync ponyfill, which will be available in es 2024.
*
Expand Down Expand Up @@ -562,7 +542,6 @@ describe('Gemini Client (client.ts)', () => {
generateContent: mockGenerateContentFn,
generateContentStream: vi.fn(),
batchEmbedContents: vi.fn(),
countTokens: vi.fn().mockResolvedValue({ totalTokens: 100 }),
} as unknown as ContentGenerator;

// Because the GeminiClient constructor kicks off an async process (startChat)
Expand Down
Loading
Loading