Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,9 @@ Use the `/model` command at any time to switch between all configured models.

You can also run models locally — no API key or cloud account needed. This is not an authentication method; instead, configure your local model endpoint in `~/.qwen/settings.json` using the `modelProviders` field.

Set `generationConfig.contextWindowSize` inside the matching provider entry
and adjust it to the context length configured on your local server.

<details>
<summary>Ollama setup</summary>

Expand All @@ -368,7 +371,10 @@ You can also run models locally — no API key or cloud account needed. This is
"id": "qwen3:32b",
"name": "Qwen3 32B (Ollama)",
"baseUrl": "http://localhost:11434/v1",
"description": "Qwen3 32B running locally via Ollama"
"description": "Qwen3 32B running locally via Ollama",
"generationConfig": {
"contextWindowSize": 131072
}
}
]
},
Expand Down Expand Up @@ -400,7 +406,10 @@ You can also run models locally — no API key or cloud account needed. This is
"id": "Qwen/Qwen3-32B",
"name": "Qwen3 32B (vLLM)",
"baseUrl": "http://localhost:8000/v1",
"description": "Qwen3 32B running locally via vLLM"
"description": "Qwen3 32B running locally via vLLM",
"generationConfig": {
"contextWindowSize": 131072
}
}
]
},
Expand Down
7 changes: 7 additions & 0 deletions docs/users/configuration/model-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,13 @@ The configuration resolution follows a strict layering model with one crucial ru
- All fields **not defined** by the provider are set to `undefined` (not inherited from settings)
- This ensures provider configurations act as a complete, self-contained "sealed package"

If a model is listed in `modelProviders`, put all model-specific
generation settings for that model in the matching provider entry. Top-level
`model.generationConfig` values, including `contextWindowSize`,
`modalities`, `customHeaders`, and `extra_body`, are ignored for provider
models. Configure those fields under
`modelProviders[authType][].generationConfig` for them to apply.

2. **When NO modelProvider model is selected** (e.g., using `--model` with a raw model ID, or using CLI/env/settings directly):
- The resolution falls through to lower layers
- Fields are populated from CLI → env → settings → defaults
Expand Down
9 changes: 7 additions & 2 deletions docs/users/configuration/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,15 +196,20 @@ To override this behavior, either set `samplingParams.max_tokens` in your settin

Overrides the default context window size for the selected model. Qwen Code determines the context window using built-in defaults based on model name matching, with a constant fallback value. Use this setting when a provider's effective context limit differs from Qwen Code's default. This value defines the model's assumed maximum context capacity, not a per-request token limit.

When the selected model is defined in `modelProviders`, set
`contextWindowSize` in that provider entry's `generationConfig` instead of the
top-level `model.generationConfig`. Provider model entries are sealed, so
top-level generation settings do not fill missing provider fields.

**modalities:**

Overrides the auto-detected input modalities for the selected model. Qwen Code automatically detects supported modalities (image, PDF, audio, video) based on model name pattern matching. Use this setting when the auto-detection is incorrect — for example, to enable `pdf` for a model that supports it but isn't recognized. Format: `{ "image": true, "pdf": true, "audio": true, "video": true }`. Omit a key or set it to `false` for unsupported types.

**customHeaders:**

Allows you to add custom HTTP headers to all API requests. This is useful for request tracing, monitoring, API gateway routing, or when different models require different headers. If `customHeaders` is defined in `modelProviders[].generationConfig.customHeaders`, it will be used directly; otherwise, headers from `model.generationConfig.customHeaders` will be used. No merging occurs between the two levels.
Allows you to add custom HTTP headers to all API requests. This is useful for request tracing, monitoring, API gateway routing, or when different models require different headers. For provider models, define `customHeaders` in `modelProviders[].generationConfig.customHeaders`. For runtime models without a matching provider entry, define it in `model.generationConfig.customHeaders`. No merging occurs between the two levels.

The `extra_body` field allows you to add custom parameters to the request body sent to the API. This is useful for provider-specific options that are not covered by the standard configuration fields. **Note: This field is only supported for OpenAI-compatible providers (`openai`, `qwen-oauth`). It is ignored for Anthropic and Gemini providers.** If `extra_body` is defined in `modelProviders[].generationConfig.extra_body`, it will be used directly; otherwise, values from `model.generationConfig.extra_body` will be used.
The `extra_body` field allows you to add custom parameters to the request body sent to the API. This is useful for provider-specific options that are not covered by the standard configuration fields. **Note: This field is only supported for OpenAI-compatible providers (`openai`, `qwen-oauth`). It is ignored for Anthropic and Gemini providers.** For provider models, define `extra_body` in `modelProviders[].generationConfig.extra_body`. For runtime models without a matching provider entry, define it in `model.generationConfig.extra_body`.

**model.openAILoggingDir examples:**

Expand Down
173 changes: 173 additions & 0 deletions packages/cli/src/utils/modelConfigUtils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -532,6 +532,179 @@ describe('modelConfigUtils', () => {
expect(result.warnings).toEqual(['Warning 1', 'Warning 2']);
});

it('should warn when top-level generationConfig fields are ignored for a provider model', () => {
const argv = {};
const modelProvider: ProviderModelConfig = {
id: 'qwen3.6-27b',
name: 'qwen3.6-27b',
baseUrl: 'http://localhost:8080/v1',
envKey: 'IK_LLAMA_API_KEY',
};
const settings = makeMockSettings({
model: {
name: 'qwen3.6-27b',
generationConfig: {
contextWindowSize: 192000,
extra_body: {
enable_thinking: true,
},
} as Record<string, unknown>,
},
modelProviders: {
[AuthType.USE_OPENAI]: [modelProvider],
},
});
const selectedAuthType = AuthType.USE_OPENAI;

vi.mocked(resolveModelConfig).mockReturnValue({
config: {
model: 'qwen3.6-27b',
apiKey: '',
baseUrl: 'http://localhost:8080/v1',
},
sources: {},
warnings: [],
});

const result = resolveCliGenerationConfig({
argv,
settings,
selectedAuthType,
});

expect(result.warnings).toHaveLength(1);
expect(result.warnings[0]).toContain(
'model.generationConfig.contextWindowSize',
);
expect(result.warnings[0]).toContain('model.generationConfig.extra_body');
expect(result.warnings[0]).toContain(
'provider model "qwen3.6-27b" from modelProviders.openai',
);
expect(result.warnings[0]).toContain(
'modelProviders.openai[].generationConfig',
);
});

it('should not warn for top-level generationConfig on runtime models', () => {
const argv = {};
const settings = makeMockSettings({
model: {
name: 'runtime-model',
generationConfig: {
contextWindowSize: 192000,
} as Record<string, unknown>,
},
modelProviders: {
[AuthType.USE_OPENAI]: [{ id: 'other-model', name: 'Other Model' }],
},
});
const selectedAuthType = AuthType.USE_OPENAI;

vi.mocked(resolveModelConfig).mockReturnValue({
config: {
model: 'runtime-model',
apiKey: '',
baseUrl: '',
contextWindowSize: 192000,
},
sources: {},
warnings: [],
});

const result = resolveCliGenerationConfig({
argv,
settings,
selectedAuthType,
});

expect(result.warnings).toEqual([]);
});

it('should not warn for generationConfig fields defined by the provider model', () => {
const argv = {};
const modelProvider: ProviderModelConfig = {
id: 'provider-model',
name: 'Provider Model',
generationConfig: {
contextWindowSize: 192000,
},
};
const settings = makeMockSettings({
model: {
name: 'provider-model',
generationConfig: {
contextWindowSize: 128000,
} as Record<string, unknown>,
},
modelProviders: {
[AuthType.USE_OPENAI]: [modelProvider],
},
});
const selectedAuthType = AuthType.USE_OPENAI;

vi.mocked(resolveModelConfig).mockReturnValue({
config: {
model: 'provider-model',
apiKey: '',
baseUrl: '',
contextWindowSize: 192000,
},
sources: {},
warnings: [],
});

const result = resolveCliGenerationConfig({
argv,
settings,
selectedAuthType,
});

expect(result.warnings).toEqual([]);
});

it('should append ignored generationConfig warnings to resolver warnings', () => {
const argv = {};
const modelProvider: ProviderModelConfig = {
id: 'provider-model',
name: 'Provider Model',
};
const settings = makeMockSettings({
model: {
name: 'provider-model',
generationConfig: {
modalities: { image: false },
} as Record<string, unknown>,
},
modelProviders: {
[AuthType.USE_OPENAI]: [modelProvider],
},
});
const selectedAuthType = AuthType.USE_OPENAI;

vi.mocked(resolveModelConfig).mockReturnValue({
config: {
model: 'provider-model',
apiKey: '',
baseUrl: '',
},
sources: {},
warnings: ['Resolver warning'],
});

const result = resolveCliGenerationConfig({
argv,
settings,
selectedAuthType,
});

expect(result.warnings).toHaveLength(2);
expect(result.warnings[0]).toBe('Resolver warning');
expect(result.warnings[1]).toContain(
'model.generationConfig.modalities is ignored',
);
expect(result.warnings[1]).toContain('Move this field');
});
Comment thread
doudouOUC marked this conversation as resolved.
Resolved

it('should use custom env when provided', () => {
const argv = {};
const settings = makeMockSettings({
Expand Down
57 changes: 56 additions & 1 deletion packages/cli/src/utils/modelConfigUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,40 @@ const AUTH_ENV_MODEL_VARS: Record<AuthType, string[]> = {
[AuthType.QWEN_OAUTH]: [],
};

function getIgnoredTopLevelGenerationConfigFields(
settingsGenerationConfig: Partial<ContentGeneratorConfig> | undefined,
modelProvider: ProviderModelConfig | undefined,
): string[] {
if (!settingsGenerationConfig || !modelProvider) {
return [];
}

const providerGenerationConfig = modelProvider.generationConfig ?? {};
return Object.keys(settingsGenerationConfig).filter(
(field) => !Object.hasOwn(providerGenerationConfig, field),
);
Comment thread
doudouOUC marked this conversation as resolved.
Resolved
}

function buildIgnoredTopLevelGenerationConfigWarning(
authType: AuthType,
modelProvider: ProviderModelConfig,
ignoredFields: string[],
): string | undefined {
if (ignoredFields.length === 0) {
return undefined;
}

const fieldList = ignoredFields
.map((field) => `model.generationConfig.${field}`)
.join(', ');
const isSingular = ignoredFields.length === 1;
const verb = isSingular ? 'is' : 'are';
const fieldReference = isSingular ? 'this field' : 'these fields';
const pronoun = isSingular ? 'it' : 'them';

return `Warning: ${fieldList} ${verb} ignored for provider model "${modelProvider.id}" from modelProviders.${authType}. Move ${fieldReference} to modelProviders.${authType}[].generationConfig for that model if you want ${pronoun} to apply.`;
}

export interface CliGenerationConfigInputs {
argv: {
model?: string | undefined;
Expand Down Expand Up @@ -188,6 +222,22 @@ export function resolveCliGenerationConfig(

Comment thread
doudouOUC marked this conversation as resolved.
Resolved
const resolved = resolveModelConfig(configSources);

// Provider-backed models are synced again during Config.refreshAuth(), which
// reapplies provider defaults after the initial resolver fallback.
const ignoredGenerationConfigWarning =
authType && modelProvider
? buildIgnoredTopLevelGenerationConfigWarning(
authType,
modelProvider,
getIgnoredTopLevelGenerationConfigFields(
settings.model?.generationConfig as
| Partial<ContentGeneratorConfig>
| undefined,
modelProvider,
),
)
: undefined;

// Resolve OpenAI logging config (CLI-specific, not part of core resolver)
const enableOpenAILogging =
(typeof argv.openaiLogging === 'undefined'
Expand All @@ -211,6 +261,11 @@ export function resolveCliGenerationConfig(
baseUrl: resolved.config.baseUrl || '',
generationConfig,
sources: resolved.sources,
warnings: resolved.warnings,
warnings: [
...resolved.warnings,
...(ignoredGenerationConfigWarning
? [ignoredGenerationConfigWarning]
: []),
],
};
}
Loading