Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1557,6 +1557,73 @@ describe('AnthropicContentGenerator', () => {
);
});

// Claude 4.8+ deprecated the `temperature` sampling parameter — the
// server responds with a 400 when it is sent. Verify the generator
// omits it for 4.8+ and keeps it for older models.
it('omits temperature on Opus 4.8 (deprecated)', async () => {
const { AnthropicContentGenerator } = await importGenerator();
anthropicState.createImpl.mockResolvedValue({
id: 'anthropic-1',
model: 'claude-opus-4-8',
content: [{ type: 'text', text: 'hi' }],
});

const generator = new AnthropicContentGenerator(
{
model: 'claude-opus-4-8',
apiKey: 'test-key',
baseUrl: 'https://api.anthropic.com',
timeout: 10_000,
maxRetries: 2,
samplingParams: { max_tokens: 500, temperature: 0.7 },
schemaCompliance: 'auto',
},
mockConfig,
);

await generator.generateContent({
model: 'models/ignored',
contents: 'Hello',
} as unknown as GenerateContentParameters);

const [anthropicRequest] =
anthropicState.lastCreateArgs as AnthropicCreateArgs;
expect(anthropicRequest).not.toHaveProperty('temperature');
});

it('keeps temperature on Opus 4.7 (still accepted)', async () => {
const { AnthropicContentGenerator } = await importGenerator();
anthropicState.createImpl.mockResolvedValue({
id: 'anthropic-1',
model: 'claude-opus-4-7',
content: [{ type: 'text', text: 'hi' }],
});

const generator = new AnthropicContentGenerator(
{
model: 'claude-opus-4-7',
apiKey: 'test-key',
baseUrl: 'https://api.anthropic.com',
timeout: 10_000,
maxRetries: 2,
samplingParams: { max_tokens: 500, temperature: 0.7 },
schemaCompliance: 'auto',
},
mockConfig,
);

await generator.generateContent({
model: 'models/ignored',
contents: 'Hello',
} as unknown as GenerateContentParameters);

const [anthropicRequest] =
anthropicState.lastCreateArgs as AnthropicCreateArgs;
expect(anthropicRequest).toEqual(
expect.objectContaining({ temperature: 0.7 }),
);
});

it("clamps effort: 'xhigh' to 'max' on Opus 4.6 (has max, lacks xhigh)", async () => {
const { AnthropicContentGenerator } = await importGenerator();
anthropicState.createImpl.mockResolvedValue({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -744,9 +744,15 @@ export class AnthropicContentGenerator implements ContentGenerator {
}
}

// Claude 4.8+ deprecated temperature — the server rejects it with a 400.
// Omit the parameter entirely for those models; older models keep the
// default of 1 (Anthropic's documented neutral value).
const temperatureValue =
getParam<number>('temperature', 'temperature') ?? 1;

return {
max_tokens: maxTokens,
temperature: getParam<number>('temperature', 'temperature') ?? 1,
...(!this.modelRejectsTemperature() && { temperature: temperatureValue }),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] This only omits temperature for 4.8+, but Anthropic's Messages API docs say temperature, top_p, and top_k should be omitted on Claude Opus 4.7 and later, including Opus 4.8. With the current request shape, claude-opus-4-7 still sends configured temperature, and claude-opus-4-8 still sends configured top_p/top_k, so both paths can continue to return 400s after this fix.

Please make this a sampling-parameter gate for the affected model set and apply it to all three fields; update the Opus 4.7 test to assert omission and add coverage for topP/topK (and the major > 4 branch). See https://platform.claude.com/docs/en/build-with-claude/working-with-messages.

— GPT-5 via Qwen Code /review

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor cleanups here — not bugs ({...false} correctly spreads to no keys):

  1. Idiom consistency + a now-dead computation. The sibling spreads just below use the ternary form (...(thinking ? { thinking } : {}), L697–698). Matching that here also lets you drop temperatureValue, which currently still runs getParam(...) even when the value is discarded:

    ...(this.modelRejectsTemperature()
      ? {}
      : { temperature: getParam<number>('temperature', 'temperature') ?? 1 }),
  2. Silent drop of an explicit temperature. A user who set temperature: 0 (e.g. for determinism) on a 4.8+ model now has it dropped with no signal. Consider a once-per-generator debugLogger.warn, latched like effortClampWarned in resolveEffectiveEffort, so the omission is visible under --debug — consistent with how effort clamping is surfaced.

top_p: getParam<number>('top_p', 'topP'),
top_k: getParam<number>('top_k', 'topK'),
};
Expand Down Expand Up @@ -858,6 +864,23 @@ export class AnthropicContentGenerator implements ContentGenerator {
return major > 4 || (major === 4 && minor >= 7);
}

/**
* Whether the model rejects the `temperature` sampling parameter with a 400.
* Claude Opus 4.8+ deprecated temperature — the server controls sampling
* determinism internally and responds with
* `"temperature is deprecated for this model."` when the parameter is sent.
* Older models (4.7 and below) and unknown/unversioned ids still accept it,
* so both return false.
*/
private modelRejectsTemperature(): boolean {
const parsed = parseClaudeModelVersion(
this.contentGeneratorConfig.model || '',
);
if (!parsed) return false;
const { major, minor } = parsed;
return major > 4 || (major === 4 && minor >= 8);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

major > 4 fires for every 5.x family — Sonnet 5, Fable 5, Mythos 5 — not just Opus, whereas the linked issue #6519 and the PR's behavior table are Opus-4.8-specific. Worth confirming Anthropic dropped temperature family-wide at 5.x.

It mirrors modelRejectsManualThinking()'s major > 4, so it reads as deliberate, and it's the safe direction (omitting avoids the 400; sending would hard-fail). The only downside if some 5.x model still accepts temperature: a user-set value is silently omitted for it. A claude-sonnet-5 test would lock in the intended 5.x behavior.

}

private buildThinkingConfig(
request: GenerateContentParameters,
effectiveEffort: ReasoningEffort | undefined,
Expand Down
Loading