Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/gemini-native-image-model-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@tanstack/ai-gemini': minor
---

Type Gemini-native image models with their own provider options. `GeminiImageModelProviderOptionsByName` mapped **every** image model to the Imagen-shaped `GeminiImageProviderOptions`, so `modelOptions: { safetySettings, thinkingConfig, imageConfig, systemInstruction }` was a compile error on `gemini-3.1-flash-image-preview`, `gemini-3.1-flash-lite-image`, `gemini-3-pro-image-preview`, and `gemini-2.5-flash-image` β€” even though those models are served by `generateContent`, whose `GenerateContentConfig` accepts all of them. The adapter compensated by forwarding only `seed`, silently dropping anything else.

The map now splits native vs Imagen, mirroring the split already used by `GeminiImageModelSizeByName` and `GeminiImageModelInputModalitiesByName`: native models get the new `GeminiNativeImageProviderOptions` (`seed`, `safetySettings`, `thinkingConfig`, `imageConfig`, `systemInstruction`), Imagen models keep `GeminiImageProviderOptions`. Both API paths now pick their config fields by name β€” never a wholesale spread β€” so neither shape's fields can reach the other's endpoint. Runtime routing moves the same way, off a `gemini-` prefix test onto membership in `GEMINI_NATIVE_IMAGE_MODELS`: a `gemini-*` image model not present in that list now routes to `generateImages` instead of `generateContent`, so it fails against that endpoint rather than silently taking the native path.

`responseModalities` stays a protected adapter default (`['TEXT', 'IMAGE']`) and is deliberately absent from the new type. `modelOptions.imageConfig` merges **over** the `imageConfig` derived from the portable `size` option, per field β€” passing only `imageConfig.imageSize` keeps the `aspectRatio` that `size` implied. `HarmCategory` and `HarmBlockThreshold` are now re-exported so `safetySettings` can be written without adding `@google/genai` to your own dependencies.

`GEMINI_NATIVE_IMAGE_MODELS` and `isGeminiNativeImageModel` are exported so callers can read the same list the adapter uses for routing.

Native `imageConfig` is now `GeminiNativeImageConfig`: only `aspectRatio` and `imageSize`. Other `@google/genai` `ImageConfig` keys type-checked and then threw on the Gemini Developer API.

**BREAKING (types only):** Imagen fields no longer compile on Gemini-native image models β€” `aspectRatio`, `negativePrompt`, `personGeneration`, `safetyFilterLevel`, `addWatermark`, `language`, `outputMimeType`, `outputCompressionQuality`, `guidanceScale`, `enhancePrompt`, `includeSafetyAttributes`, `includeRaiReason`, `outputGcsUri`, `labels`. They previously type-checked but were already dropped at runtime (only `seed` was ever forwarded to `generateContent`), so no request behaviour changes. The compiler now reports what was already happening. Migrate `aspectRatio` to the portable `size` option (`'16:9_4K'`) or to `modelOptions.imageConfig`, and drop the rest. Native `imageConfig` also no longer accepts Vertex-only SDK keys such as `personGeneration` and `outputMimeType`. `GeminiImageAdapter.generateImages` (and its `~types.providerOptions`) also widens from `ImageGenerationOptions<GeminiImageProviderOptions>` to `ImageGenerationOptions<GeminiAnyImageProviderOptions>`, which affects code structurally annotated against the old signature.
34 changes: 32 additions & 2 deletions docs/adapters/gemini.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,10 @@ The Gemini adapter supports two types of image generation:
- **Gemini native image models** (NanoBanana) β€” Use the `generateContent` API with models like `gemini-3.1-flash-image`. These support aspect ratio control plus resolution tiers (`512`, `1K`, `2K`, `4K`); which ratios and tiers are accepted varies per model and is enforced at compile time.
- **Imagen models** β€” Use the `generateImages` API with models like `imagen-4.0-generate-001`. These are dedicated image generation models with WIDTHxHEIGHT sizing.

The adapter automatically routes to the correct API based on the model name β€” models starting with `gemini-` use `generateContent`, while `imagen-` models use `generateImages`.
The adapter routes to `generateContent` when the model is in
`GEMINI_NATIVE_IMAGE_MODELS`. Imagen models, and any id this package does not
know, use `generateImages`. Import the list or `isGeminiNativeImageModel`
from `@tanstack/ai-gemini`.

### Example: Gemini Native Image Generation (NanoBanana)

Expand Down Expand Up @@ -502,6 +505,10 @@ const result = await generateImage({

### Image Model Options

`modelOptions` is typed per model family, because the two families hit different APIs.

Imagen models (`generateImages`) take `GenerateImagesConfig` fields:

Comment thread
coderabbitai[bot] marked this conversation as resolved.
```typescript ignore
import { generateImage } from "@tanstack/ai";
import { geminiImage } from "@tanstack/ai-gemini";
Expand All @@ -517,6 +524,29 @@ const result = await generateImage({
});
```

Gemini native models (`generateContent`) take `seed`, `safetySettings`,
`thinkingConfig`, `imageConfig`, and `systemInstruction`.
`imageConfig` accepts only `aspectRatio` and `imageSize` on the Gemini
Developer API.

```typescript
import { generateImage } from "@tanstack/ai";
import { geminiImage } from "@tanstack/ai-gemini";

const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image"),
prompt: "...",
size: "16:9_4K",
modelOptions: {
thinkingConfig: { thinkingBudget: 512 },
// Merged over the imageConfig derived from `size`, per field.
imageConfig: { imageSize: "2K" },
},
});
```

See [Image Generation](../media/image-generation) for the full native option list.

## Text-to-Speech (Experimental)

> **Note:** Gemini TTS is experimental and may require the Live API for full functionality.
Expand Down Expand Up @@ -604,7 +634,7 @@ Creates a Gemini summarization adapter.

### `geminiImage(model, config?)` / `createGeminiImage(model, apiKey, config?)`

Creates a Gemini image adapter. Automatically routes to the correct API based on the model name β€” `gemini-*` models use `generateContent`, `imagen-*` models use `generateImages`.
Creates a Gemini image adapter. Models in `GEMINI_NATIVE_IMAGE_MODELS` use `generateContent`. Imagen models, and any unknown id, use `generateImages`.

### `geminiSpeech(model, config?)` / `createGeminiSpeech(model, apiKey, config?)`

Expand Down
4 changes: 2 additions & 2 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,7 @@
"label": "Image Generation",
"to": "media/image-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-14"
"updatedAt": "2026-08-18"
},
{
"label": "Video Generation",
Expand Down Expand Up @@ -811,7 +811,7 @@
"label": "Google Gemini",
"to": "adapters/gemini",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-14"
"updatedAt": "2026-08-18"
},
{
"label": "Ollama",
Expand Down
38 changes: 37 additions & 1 deletion docs/media/image-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -642,7 +642,7 @@ const result = await generateImage({

#### Gemini Native Model Options (NanoBanana)

Gemini native image models accept `GenerateContentConfig` options directly in `modelOptions`:
Gemini native image models are served by `generateContent`, so their `modelOptions` are `GenerateContentConfig` fields β€” a different shape from the Imagen options above:

```typescript
import { generateImage } from "@tanstack/ai";
Expand All @@ -652,9 +652,45 @@ const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image"),
prompt: "A beautiful garden",
size: "16:9_4K",
modelOptions: {
seed: 42,
thinkingConfig: { thinkingBudget: 512 },
systemInstruction: "Always render in watercolor.",
// Merged over the imageConfig derived from `size`, per field. This keeps
// the 16:9 aspect ratio and overrides only the resolution tier.
// imageConfig accepts only aspectRatio and imageSize on the Gemini
// Developer API.
imageConfig: { imageSize: "2K" },
},
});
```

`safetySettings` takes the SDK's `HarmCategory` / `HarmBlockThreshold` enums, so plain strings won't type-check. Both are re-exported from `@tanstack/ai-gemini` β€” you don't need `@google/genai` in your own dependencies:

```typescript
import { generateImage } from "@tanstack/ai";
import {
HarmBlockThreshold,
HarmCategory,
geminiImage,
} from "@tanstack/ai-gemini";

const result = await generateImage({
adapter: geminiImage("gemini-3.1-flash-image-preview"),
prompt: "A beautiful garden",
modelOptions: {
safetySettings: [
{
category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
threshold: HarmBlockThreshold.BLOCK_ONLY_HIGH,
},
],
},
});
```

`responseModalities` is not accepted β€” the adapter always requests `['TEXT', 'IMAGE']`, so nothing can silently disable image output.

### Response Format

The image generation result includes:
Expand Down
124 changes: 90 additions & 34 deletions packages/ai-gemini/src/adapters/image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
} from '../utils'
import { buildGeminiUsage } from '../usage'
import {
isGeminiNativeImageModel,
parseNativeImageSize,
sizeToAspectRatio,
validateImageSize,
Expand All @@ -15,10 +16,11 @@ import {
} from '../image/image-provider-options'
import type { GeminiImageModels } from '../model-meta'
import type {
GeminiAnyImageProviderOptions,
GeminiImageModelInputModalitiesByName,
GeminiImageModelProviderOptionsByName,
GeminiImageModelSizeByName,
GeminiImageProviderOptions,
GeminiNativeImageProviderOptions,
} from '../image/image-provider-options'
import type {
GeneratedImage,
Expand All @@ -35,6 +37,7 @@ import type {
GenerateImagesConfig,
GenerateImagesResponse,
GoogleGenAI,
ImageConfig,
Part,
} from '@google/genai'
import type { GeminiClientConfig } from '../utils/client'
Expand Down Expand Up @@ -65,7 +68,7 @@ export class GeminiImageAdapter<
TModel extends GeminiImageModel,
> extends BaseImageAdapter<
TModel,
GeminiImageProviderOptions,
GeminiAnyImageProviderOptions,
GeminiImageModelProviderOptionsByName,
GeminiImageModelSizeByName,
GeminiImageModelInputModalitiesByName
Expand All @@ -75,7 +78,7 @@ export class GeminiImageAdapter<

// Type-only property - never assigned at runtime
declare '~types': {
providerOptions: GeminiImageProviderOptions
providerOptions: GeminiAnyImageProviderOptions
modelProviderOptionsByName: GeminiImageModelProviderOptionsByName
modelSizeByName: GeminiImageModelSizeByName
modelInputModalitiesByName: GeminiImageModelInputModalitiesByName
Expand All @@ -89,7 +92,7 @@ export class GeminiImageAdapter<
}

async generateImages(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiAnyImageProviderOptions>,
): Promise<ImageGenerationResult> {
const { model, logger } = options

Expand Down Expand Up @@ -121,7 +124,7 @@ export class GeminiImageAdapter<
)
}

if (this.isGeminiImageModel(model)) {
if (isGeminiNativeImageModel(model)) {
return await this.generateWithGeminiApi(options, resolved)
}

Expand Down Expand Up @@ -155,29 +158,40 @@ export class GeminiImageAdapter<
}
}

private isGeminiImageModel(model: string): boolean {
return model.startsWith('gemini-')
}

private async generateWithGeminiApi(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiNativeImageProviderOptions>,
resolved: ResolvedMediaPrompt,
): Promise<ImageGenerationResult> {
const { model, size, numberOfImages, modelOptions } = options

const parsedSize = size ? parseNativeImageSize(size) : undefined

// GeminiImageProviderOptions is Imagen-shaped β€” most fields
// (personGeneration, safetyFilterLevel, addWatermark, outputMimeType,
// outputCompressionQuality, guidanceScale, enhancePrompt,
// includeSafetyAttributes, includeRaiReason, outputGcsUri, labels,
// negativePrompt, language) are only valid on GenerateImagesConfig and
// would be rejected by the Gemini-native generateContent path. Pick only
// the fields that are valid on GenerateContentConfig instead of spreading
// the whole options object.
const nativeConfig: GenerateContentConfig = {}
if (modelOptions?.seed !== undefined) {
nativeConfig.seed = modelOptions.seed
// The portable `size` option is the baseline; modelOptions.imageConfig is
// the provider escape hatch and wins per field, so a caller passing only
// `imageConfig.imageSize` keeps the aspectRatio derived from `size`.
const imageConfig: ImageConfig = {
...(parsedSize?.aspectRatio && { aspectRatio: parsedSize.aspectRatio }),
...(parsedSize?.resolution && { imageSize: parsedSize.resolution }),
...modelOptions?.imageConfig,
}

// Named picks, never a wholesale spread: the Imagen-shaped fields of
// GeminiImageProviderOptions (personGeneration, safetyFilterLevel,
// addWatermark, outputMimeType, …) are only valid on GenerateImagesConfig
// and would be rejected by generateContent. Picking by name means no
// Imagen field can reach this path even if one slips past the per-model
// provider-options map.
const nativeConfig: GenerateContentConfig = {
...(modelOptions?.seed !== undefined && { seed: modelOptions.seed }),
...(modelOptions?.safetySettings !== undefined && {
safetySettings: modelOptions.safetySettings,
}),
...(modelOptions?.thinkingConfig !== undefined && {
thinkingConfig: modelOptions.thinkingConfig,
}),
...(modelOptions?.systemInstruction !== undefined && {
systemInstruction: modelOptions.systemInstruction,
}),
}

const config: GenerateContentConfig = {
Expand All @@ -186,16 +200,7 @@ export class GeminiImageAdapter<
// IMPORTANT: responseModalities is a protected default β€” set it AFTER
// nativeConfig so nothing can silently disable image output.
responseModalities: ['TEXT', 'IMAGE'],
...(parsedSize && {
imageConfig: {
...(parsedSize.aspectRatio && {
aspectRatio: parsedSize.aspectRatio,
}),
...(parsedSize.resolution && {
imageSize: parsedSize.resolution,
}),
},
}),
...(Object.keys(imageConfig).length > 0 && { imageConfig }),
}

const contents = this.buildContents(resolved, numberOfImages)
Expand Down Expand Up @@ -321,19 +326,70 @@ export class GeminiImageAdapter<
}

private buildImagenConfig(
options: ImageGenerationOptions<GeminiImageProviderOptions>,
options: ImageGenerationOptions<GeminiAnyImageProviderOptions>,
): GenerateImagesConfig {
const { size, numberOfImages, modelOptions } = options

// Build with conditional spreads β€” under exactOptionalPropertyTypes the
// vendor `GenerateImagesConfig` fields are `field?: T` (no `| undefined`),
// so we can only assign the property when we actually have a value.
const sizeAspectRatio = size ? sizeToAspectRatio(size) : undefined

// Named picks, never a wholesale spread β€” the mirror image of the native
// path below. A native-only field (safetySettings, thinkingConfig,
// imageConfig, systemInstruction) belongs to GenerateContentConfig and is
// rejected by generateImages with 400 INVALID_ARGUMENT, so it must not be
// able to reach here even when the caller's `modelOptions` was typed
// against both shapes at once (e.g. an adapter inferred from a union of
// model names).
return {
numberOfImages: numberOfImages ?? 1,
// Map size to aspect ratio if provided (modelOptions.aspectRatio will override)
// Map size to aspect ratio if provided; modelOptions.aspectRatio,
// picked after it, overrides.
...(sizeAspectRatio !== undefined && { aspectRatio: sizeAspectRatio }),
...modelOptions,
...(modelOptions?.aspectRatio !== undefined && {
aspectRatio: modelOptions.aspectRatio,
}),
...(modelOptions?.personGeneration !== undefined && {
personGeneration: modelOptions.personGeneration,
}),
...(modelOptions?.safetyFilterLevel !== undefined && {
safetyFilterLevel: modelOptions.safetyFilterLevel,
}),
...(modelOptions?.seed !== undefined && { seed: modelOptions.seed }),
...(modelOptions?.addWatermark !== undefined && {
addWatermark: modelOptions.addWatermark,
}),
...(modelOptions?.language !== undefined && {
language: modelOptions.language,
}),
...(modelOptions?.negativePrompt !== undefined && {
negativePrompt: modelOptions.negativePrompt,
}),
...(modelOptions?.outputMimeType !== undefined && {
outputMimeType: modelOptions.outputMimeType,
}),
...(modelOptions?.outputCompressionQuality !== undefined && {
outputCompressionQuality: modelOptions.outputCompressionQuality,
}),
...(modelOptions?.guidanceScale !== undefined && {
guidanceScale: modelOptions.guidanceScale,
}),
...(modelOptions?.enhancePrompt !== undefined && {
enhancePrompt: modelOptions.enhancePrompt,
}),
...(modelOptions?.includeSafetyAttributes !== undefined && {
includeSafetyAttributes: modelOptions.includeSafetyAttributes,
}),
...(modelOptions?.includeRaiReason !== undefined && {
includeRaiReason: modelOptions.includeRaiReason,
}),
...(modelOptions?.outputGcsUri !== undefined && {
outputGcsUri: modelOptions.outputGcsUri,
}),
...(modelOptions?.labels !== undefined && {
labels: modelOptions.labels,
}),
}
}

Expand Down
Loading
Loading