Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions docs/design/2026-09-03-outbound-session-id-header.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Routify `session_id` Header

## Summary

Qwen Code attaches its current session ID as the `session_id` HTTP header only when the outbound LLM request hostname is one of the three Routify endpoints documented by ModelRouter: `routify.alibaba-inc.com`, `routify-online.alibaba-inc.com`, or `routify-pub.alibaba-inc.com`.

The behavior is intentionally not configurable. Qwen Code does not attach the header to subdomains, other `alibaba-inc.com` hosts, or non-Routify providers. Standard fetch redirect behavior applies after the initial destination check, so a Routify response can forward the header by redirecting the request.

## Motivation

Routify's ModelRouter accepts `session_id` as a session-affinity and traffic-marking value. Qwen Code already maintains a session ID, but it is currently local metadata and never reaches the ModelRouter request. Reusing it gives Routify one stable affinity value per CLI session without creating another identifier.

## Security boundary

A session ID is a stable cross-request identifier. The implementation therefore requires HTTPS and compares the parsed request hostname to a fixed set of three ModelRouter hostnames. It does not use suffix matching, wildcards, path matching, or a user-configurable allowlist. Invalid URLs fail closed.

The Qwen Code session ID replaces any custom `session_id` value on an eligible request so the affinity marker cannot disagree with the active session. All other existing headers, including authorization, are preserved.

## Request lifecycle

OpenAI-compatible and Anthropic clients receive a fetch wrapper. The wrapper reads `Config.getSessionId()` immediately before each HTTP request. This matters because `/clear` starts a new session without rebuilding the SDK client.

Gemini requests use the SDK's request-level `httpOptions.headers`. The header is rebuilt for generate, streaming generate, and embedding requests. Gemini injection requires an explicit Routify `baseUrl`; implicit SDK endpoints remain unchanged.

## Provider coverage

- The default OpenAI-compatible provider covers Routify's OpenAI protocol and subclasses that inherit its client construction.
- DashScope has a separate client constructor and is integrated explicitly.
- Anthropic uses the same per-request fetch wrapper.
- Gemini and Vertex use request-level HTTP options when the base URL points to Routify.

Non-LLM traffic, other domains, MCP requests, tool fetches, subprocesses, `traceparent`, request IDs, and body metadata are out of scope.

## Verification

Unit tests cover exact-host and HTTPS matching, rejection of lookalike hosts, invalid URLs, preservation and precedence of combined `Request` and init headers, empty values, session rotation, and the shared runtime-fetch wrapper. Provider tests verify the OpenAI-compatible construction paths install a working correlation layer, and Gemini tests cover constructor destinations, generation, embedding, and successive requests observing a changed session ID.
30 changes: 21 additions & 9 deletions docs/developers/development/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,15 +393,27 @@ Verify both flags when wiring an ARMS+DashScope correlation setup:
}
```

### Other outbound correlation headers

`X-Qwen-Code-Session-Id` and `X-Qwen-Code-Request-Id` are **not part of
this PR**. They will be designed and proposed in their own follow-up
PR(s) under the same `outboundCorrelation.*` namespace, each with its
own threat model and operator-consent flow. PR #4390 review (LaZzyMan)
established the principle: "telemetry's scope of work doesn't include
sending identifiers to LLM providers"; correlation-header work moves to
its own design discussion rather than landing under telemetry.
### Routify session affinity

Qwen Code's LLM requests through the OpenAI-compatible, DashScope, Anthropic,
Gemini, and Vertex provider paths include the current Qwen Code session ID in
the `session_id` header when addressed over HTTPS directly to `routify.alibaba-inc.com`,
`routify-online.alibaba-inc.com`, or `routify-pub.alibaba-inc.com`. Routify's
ModelRouter uses this value for session affinity and traffic marking. This
behavior is not controlled by `telemetry.enabled` or
`outboundCorrelation.*`.

The initial destination match is deliberately narrow: Qwen Code does not
attach the header to subdomains, other `alibaba-inc.com` hosts, or other LLM
endpoints. The standard fetch redirect behavior still applies after that
match, so a Routify response can forward the header by redirecting the request.

The session ID is read for every request, so a new session created by
`/clear` gets a new affinity value without rebuilding the SDK client. Gemini
requires an explicit Routify `baseUrl` so Qwen Code can verify the
destination.

`X-Qwen-Code-Request-Id` is not implemented.

## Inbound correlation (daemon HTTP API)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,54 @@ describe('AnthropicContentGenerator', () => {
expect(headers['x-app']).toBe('cli');
expect(anthropicState.constructorOptions?.['authToken']).toBe('test-key');
expect(anthropicState.constructorOptions?.['apiKey']).toBeNull();
expect(anthropicState.constructorOptions?.['fetch']).toEqual(
Comment thread
DragonnZhang marked this conversation as resolved.
expect.any(Function),
);
});

it('installs session ID injection on the runtime fetch', async () => {
const runtimeFetch = vi.fn(
async (_input: string | URL | Request, _init?: RequestInit) =>
new Response(),
);
vi.doMock('../../utils/runtimeFetchOptions.js', async (importOriginal) => {
const actual =
await importOriginal<
typeof import('../../utils/runtimeFetchOptions.js')
>();
return {
...actual,
buildRuntimeFetchOptions: vi.fn(() => ({ fetch: runtimeFetch })),
};
});

try {
const { AnthropicContentGenerator } = await importGenerator();
void new AnthropicContentGenerator(
{
model: 'claude-test',
apiKey: 'test-key',
baseUrl: 'https://routify-pub.alibaba-inc.com/protocol/anthropic',
timeout: 10_000,
maxRetries: 2,
samplingParams: {},
schemaCompliance: 'auto',
},
mockConfig,
);

const sessionAwareFetch = anthropicState.constructorOptions?.[
'fetch'
] as typeof fetch;
await sessionAwareFetch(
'https://routify-pub.alibaba-inc.com/protocol/anthropic/v1',
);

const headers = new Headers(runtimeFetch.mock.calls[0][1]?.headers);
expect(headers.get('session_id')).toBe('test-session');
} finally {
vi.doUnmock('../../utils/runtimeFetchOptions.js');
}
});

it('uses QwenCode identity + apiKey auth when baseURL is api.anthropic.com', async () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* SPDX-License-Identifier: Apache-2.0
*/

import Anthropic from '@anthropic-ai/sdk';
import Anthropic, { type ClientOptions } from '@anthropic-ai/sdk';
import type {
EmbedContentParameters,
EmbedContentResponse,
Expand All @@ -27,6 +27,7 @@ type MessageCreateParamsNonStreaming =
Anthropic.MessageCreateParamsNonStreaming;
type MessageCreateParamsStreaming = Anthropic.MessageCreateParamsStreaming;
type RawMessageStreamEvent = Anthropic.RawMessageStreamEvent;
type AnthropicFetch = NonNullable<ClientOptions['fetch']>;
import { AnthropicContentConverter } from './converter.js';
import { buildAnthropicUsageMetadata } from './usage.js';
import {
Expand All @@ -53,6 +54,7 @@ import { setToolCallPreparations } from '../tool-call-preparation.js';
import { InvalidStreamError } from '../invalid-stream-error.js';
import { parseToolCallArguments } from '../tool-call-arguments.js';
import { classifyRetryError } from '../../utils/retryErrorClassification.js';
import { buildSessionAwareFetch } from '../outbound-session-id.js';
import { isRetryableStreamTransportError } from '../stream-transport-retry.js';
import {
reportAnthropicEvent,
Expand Down Expand Up @@ -332,7 +334,6 @@ export class AnthropicContentGenerator implements ContentGenerator {
'anthropic',
this.cliConfig.getProxy(),
);

// IdeaLab-style Anthropic proxies expect `Authorization: Bearer <token>`
// instead of the SDK-default `x-api-key` header. Use the SDK's
// `authToken` parameter (sends `Authorization: Bearer` natively) only
Expand Down Expand Up @@ -360,6 +361,10 @@ export class AnthropicContentGenerator implements ContentGenerator {
maxRetries: contentGeneratorConfig.maxRetries,
defaultHeaders,
...runtimeOptions,
fetch: buildSessionAwareFetch(
runtimeOptions.fetch,
this.cliConfig,
) as unknown as AnthropicFetch,
});

this.converter = new AnthropicContentConverter(
Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/core/llm-content-generator/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ describe('createLlmContentGenerator', () => {
}),
}),
config,
mockConfig,
);
});

Expand All @@ -82,6 +83,7 @@ describe('createLlmContentGenerator', () => {
}),
}),
config,
mockConfig,
);
expect(vi.mocked(LlmContentGenerator).mock.calls[0]?.[0]).not.toEqual(
expect.objectContaining({
Expand Down
1 change: 1 addition & 0 deletions packages/core/src/core/llm-content-generator/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ export function createLlmContentGenerator(
httpOptions,
},
config,
gcConfig,
);

return llmContentGenerator;
Expand Down
Loading
Loading