Skip to content

fix(billing): split image input tokens for gpt-image-2 #2176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
steebchen merged 8 commits into from
May 7, 2026
Merged

fix(billing): split image input tokens for gpt-image-2 #2176

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
59c4efb
fix(billing): split image input tokens for gpt-image-2
steebchen May 6, 2026
913926b
merge: sync with origin/main, resolve billing conflicts
steebchen-bot May 7, 2026
2d29c96
fix: add null args for image tokens in existing callers
steebchen-bot May 7, 2026
44729de
test(billing): read azure discount from model def
steebchen May 7, 2026
c40ab32
fix(models): zero text output price for gpt-image-2
steebchen May 7, 2026
75d6d88
fix(billing): split text- vs image-cached pricing for gpt-image-2
steebchen May 7, 2026
feb9cbd
Merge remote-tracking branch 'origin/main' into bill-gpt-image-2-inpu…
steebchen May 7, 2026
b608d99
test(billing): add image-token args to bedrock cache test
steebchen May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions apps/gateway/src/chat/chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3598,6 +3598,8 @@ chat.openapi(completions, async (c) => {
null, // webSearchCount
project.organizationId,
undefined,
null,
null,
{
cacheWriteTokens,
cacheWrite1hTokens,
Expand Down Expand Up @@ -3758,6 +3760,8 @@ chat.openapi(completions, async (c) => {
null, // webSearchCount
project.organizationId,
undefined,
null,
null,
{
cacheWriteTokens:
cachedResponse.usage?.prompt_tokens_details?.cache_write_tokens ??
Expand Down Expand Up @@ -6149,6 +6153,8 @@ chat.openapi(completions, async (c) => {
webSearchCount,
project.organizationId,
image_config?.image_quality,
null,
null,
{
cacheWriteTokens: cacheCreationTokens,
cacheWrite1hTokens: cacheCreation1hTokens,
Expand Down Expand Up @@ -7437,6 +7443,8 @@ chat.openapi(completions, async (c) => {
webSearchCount,
project.organizationId,
image_config?.image_quality,
null,
null,
{
cacheWriteTokens: cacheCreationTokens,
cacheWrite1hTokens: cacheCreation1hTokens,
Expand Down Expand Up @@ -7751,6 +7759,8 @@ chat.openapi(completions, async (c) => {
webSearchCount,
project.organizationId,
image_config?.image_quality,
null,
null,
{
cacheWriteTokens: cacheCreationTokens,
cacheWrite1hTokens: cacheCreation1hTokens,
Expand Down Expand Up @@ -9441,6 +9451,8 @@ chat.openapi(completions, async (c) => {
webSearchCount,
project.organizationId,
image_config?.image_quality,
imageInputTokens,
imageOutputTokens,
{
cacheWriteTokens: cacheCreationTokens,
cacheWrite1hTokens: cacheCreation1hTokens,
Expand Down
104 changes: 103 additions & 1 deletion apps/gateway/src/lib/costs.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { beforeEach, describe, expect, it, vi } from "vitest";

import { models } from "@llmgateway/models";

import { calculateCosts } from "./costs.js";

const { mockGetEffectiveDiscount } = vi.hoisted(() => ({
Expand Down Expand Up @@ -126,6 +128,8 @@ describe("calculateCosts", () => {
null,
null,
undefined,
null,
null,
{
cacheWriteTokens: 1659,
},
Expand Down Expand Up @@ -160,6 +164,8 @@ describe("calculateCosts", () => {
null,
null,
undefined,
null,
null,
{
cacheWriteTokens: 1000,
cacheWrite1hTokens: 700,
Expand Down Expand Up @@ -192,6 +198,8 @@ describe("calculateCosts", () => {
null,
null,
undefined,
null,
null,
{
cacheWriteTokens: 1000,
},
Expand Down Expand Up @@ -450,7 +458,7 @@ describe("calculateCosts", () => {
"low",
);

const expectedInputCost = 1000 * (8 / 1e6);
const expectedInputCost = 1000 * (5 / 1e6);
const expectedImageOutputCost = 2000 * (30 / 1e6);

expect(result.imageOutputTokens).toBe(2000);
Expand All @@ -462,6 +470,100 @@ describe("calculateCosts", () => {
);
});

it("should bill reported image input tokens at imageInputPrice for gpt-image-2 edits", async () => {
// /v1/images/edits sends input images as part of the prompt. OpenAI's
// usage payload reports text vs image tokens via input_tokens_details.
// We expect the gateway to bill the image portion at imageInputPrice
// ($8/M) and the remaining text portion at inputPrice ($5/M) — without
// double-billing image tokens at the text rate.
const promptTokens = 524; // 12 text + 512 image (from real OpenAI response)
const reportedImageInputTokens = 512;
const completionTokens = 196;
const reportedImageOutputTokens = 196;

const result = await calculateCosts(
"gpt-image-2",
"openai",
promptTokens,
completionTokens,
null, // cachedTokens
undefined, // fullOutput
null, // reasoningTokens
1, // outputImageCount
"1024x1024", // imageSize
0, // inputImageCount (not used for openai)
null, // webSearchCount
null, // organizationId
"low", // imageQuality
reportedImageInputTokens,
reportedImageOutputTokens,
);

const expectedTextInputCost =
(promptTokens - reportedImageInputTokens) * (5 / 1e6);
const expectedImageInputCost = reportedImageInputTokens * (8 / 1e6);
const expectedImageOutputCost = reportedImageOutputTokens * (30 / 1e6);

expect(result.imageInputTokens).toBe(reportedImageInputTokens);
expect(result.imageInputCost).toBeCloseTo(expectedImageInputCost);
expect(result.imageOutputTokens).toBe(reportedImageOutputTokens);
expect(result.imageOutputCost).toBeCloseTo(expectedImageOutputCost);
expect(result.inputCost).toBeCloseTo(
expectedTextInputCost + expectedImageInputCost,
);
expect(result.outputCost).toBeCloseTo(expectedImageOutputCost);
expect(result.totalCost).toBeCloseTo(
expectedTextInputCost + expectedImageInputCost + expectedImageOutputCost,
);
});

it("should apply azure discount on top of split image/text input pricing for gpt-image-2", async () => {
const promptTokens = 524;
const reportedImageInputTokens = 512;
const completionTokens = 196;
const reportedImageOutputTokens = 196;

const result = await calculateCosts(
"gpt-image-2",
"azure",
promptTokens,
completionTokens,
null,
undefined,
null,
1,
"1024x1024",
0,
null,
null,
"low",
reportedImageInputTokens,
reportedImageOutputTokens,
);

// Read discount from the model definition so the test stays correct
// even if the azure discount value changes.
const azureProvider = models
.find((m) => m.id === "gpt-image-2")
?.providers.find((p) => p.providerId === "azure");
const discountMultiplier = 1 - (azureProvider?.discount ?? 0);
Comment on lines +584 to +587

@coderabbitai coderabbitai Bot May 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Guard against false positives by asserting Azure model mapping exists.
If the lookup fails, discountMultiplier becomes 1 and this test no longer proves Azure-discount behavior.

💡 Proposed test hardening 
 		const azureProvider = models
 			.find((m) => m.id === "gpt-image-2")
 			?.providers.find((p) => p.providerId === "azure");
+		expect(azureProvider).toBeDefined();
 		const discountMultiplier = 1 - (azureProvider?.discount ?? 0);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const azureProvider = models
.find((m) => m.id === "gpt-image-2")
?.providers.find((p) => p.providerId === "azure");
const discountMultiplier = 1 - (azureProvider?.discount ?? 0);
const azureProvider = models
.find((m) => m.id === "gpt-image-2")
?.providers.find((p) => p.providerId === "azure");
expect(azureProvider).toBeDefined();
const discountMultiplier = 1 - (azureProvider?.discount ?? 0);
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/gateway/src/lib/costs.spec.ts` around lines 546 - 549, The test
currently computes discountMultiplier from a possibly undefined azureProvider,
which yields a false-positive when the lookup fails; update the test around the
models lookup so you assert that the Azure mapping was found (e.g.,
assert/expect that azureProvider is defined) before computing
discountMultiplier, and fail the test if it's missing so the test genuinely
verifies Azure-discount behavior (reference the azureProvider, models lookup,
and discountMultiplier identifiers when making the change).

const expectedTextInputCost =
(promptTokens - reportedImageInputTokens) *
(5 / 1e6) *
discountMultiplier;
const expectedImageInputCost =
reportedImageInputTokens * (8 / 1e6) * discountMultiplier;
const expectedImageOutputCost =
reportedImageOutputTokens * (30 / 1e6) * discountMultiplier;

expect(result.imageInputTokens).toBe(reportedImageInputTokens);
expect(result.imageInputCost).toBeCloseTo(expectedImageInputCost);
expect(result.inputCost).toBeCloseTo(
expectedTextInputCost + expectedImageInputCost,
);
expect(result.outputCost).toBeCloseTo(expectedImageOutputCost);
});
Comment on lines +558 to +603

@coderabbitai coderabbitai Bot May 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Azure test is missing imageOutputTokens, imageOutputCost, and totalCost assertions.

The companion OpenAI test (lines 507–517) checks all six cost breakdown fields plus totalCost, but the Azure variant only checks imageInputTokens, imageInputCost, inputCost, and outputCost. This leaves the discount-applied imageOutputCost and the aggregate totalCost unverified — which is exactly what the PR description says this test exists to confirm ("the gpt-image-2 image output portion uses reportedImageOutputTokens for its discounted image-output cost").

🛡️ Proposed additions 
 		expect(result.imageInputTokens).toBe(reportedImageInputTokens);
 		expect(result.imageInputCost).toBeCloseTo(expectedImageInputCost);
 		expect(result.inputCost).toBeCloseTo(
 			expectedTextInputCost + expectedImageInputCost,
 		);
 		expect(result.outputCost).toBeCloseTo(expectedImageOutputCost);
+		expect(result.imageOutputTokens).toBe(reportedImageOutputTokens);
+		expect(result.imageOutputCost).toBeCloseTo(expectedImageOutputCost);
+		expect(result.totalCost).toBeCloseTo(
+			expectedTextInputCost + expectedImageInputCost + expectedImageOutputCost,
+		);
 	});
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/gateway/src/lib/costs.spec.ts` around lines 520 - 565, The Azure image
test "should apply azure discount on top of split image/text input pricing for
gpt-image-2" is missing assertions for imageOutputTokens, imageOutputCost, and
totalCost; update the spec to assert that result.imageOutputTokens equals
reportedImageOutputTokens, result.imageOutputCost is close to
expectedImageOutput (use the same discountMultiplier and per-token rate used to
compute expectedImageOutputCost), and result.totalCost equals the sum of
inputCost and outputCost (or assert closeTo expectedTextInputCost +
expectedImageInputCost + expectedImageOutputCost). Reference the test name,
calculateCosts call, and result properties imageOutputTokens, imageOutputCost,
and totalCost when adding these assertions.


it("should return null for all image fields when no images", async () => {
const result = await calculateCosts("gpt-4", "openai", 100, 50, null);

Expand Down
50 changes: 39 additions & 11 deletions apps/gateway/src/lib/costs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,8 @@ export async function calculateCosts(
webSearchCount: number | null = null,
organizationId: string | null = null,
Comment on lines 115 to 116

@chatgpt-codex-connector chatgpt-codex-connector Bot May 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass image token details when billing cached responses

Because the new split pricing depends on reportedImageInputTokens/reportedImageOutputTokens, cached non-streaming responses still get billed incorrectly: the cache-hit call at apps/gateway/src/chat/chat.ts:3716 recalculates from cachedResponse.usage but never passes prompt_tokens_details.image_tokens or completion_tokens_details.image_tokens (and uses outputImageCount = 0). For cached gpt-image-2 edits/generations, this means image input tokens fall back to the new $5/M text rate and image output tokens are not charged at the $30/M image rate, even though those details are stored in the transformed cached response.

Useful? React with 👍 / 👎.

imageQuality?: string,
reportedImageInputTokens: number | null = null,
reportedImageOutputTokens: number | null = null,
options?: {
cacheWriteTokens?: number | null;
cacheWrite1hTokens?: number | null;
Expand Down Expand Up @@ -330,18 +332,30 @@ export async function calculateCosts(
return byResolution[size ?? "default"] ?? byResolution["default"];
}

// Track image input tokens separately (for Google image generation models).
// Uses imageInputTokensByResolution for per-resolution token counts and
// imageInputPrice for the per-token price. Falls back to 560 tokens/image
// with imageInputPrice if no resolution map is present.
// Track image input tokens separately. For image-output models (e.g.
// gpt-image-2) we prefer the upstream-reported image_tokens count, since
// the provider tokenises the input image and bills against it directly.
// For Google image-generation models we fall back to inputImageCount *
// imageInputTokensByResolution[size] (or 560/image legacy default).
const imageInputTokensPerImage = resolveTokensPerImage(
providerInfo.imageInputTokensByResolution,
imageSize,
);
const imageInputPricePerToken = providerInfo.imageInputPrice;
const isImageOutputModel = modelInfo.output?.includes("image") ?? false;
let imageInputTokens: number | null = null;
let imageInputCost: Decimal | null = null;
if (imageInputPricePerToken && inputImageCount > 0) {
if (
imageInputPricePerToken &&
isImageOutputModel &&
reportedImageInputTokens &&
reportedImageInputTokens > 0
Comment on lines +351 to +352
) {
imageInputTokens = reportedImageInputTokens;
imageInputCost = new Decimal(imageInputTokens)
.times(imageInputPricePerToken)
.times(discountMultiplier);

@chatgpt-codex-connector chatgpt-codex-connector Bot May 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid charging cached image tokens at full input rate

When OpenAI/Azure returns input_tokens_details.cached_tokens together with image_tokens, cached input is subtracted from the text prompt path but imageInputCost still multiplies the full reported image-token count by the full image input rate; those cached image tokens are then also charged again via cachedInputCost. For repeated gpt-image-2 image edits where the cached tokens include image input tokens, this overcharges instead of applying the cached-input rate, so allocate/subtract cached tokens from the image-token portion before multiplying by imageInputPrice.

Useful? React with 👍 / 👎.

} else if (imageInputPricePerToken && inputImageCount > 0) {
const LEGACY_TOKENS_PER_INPUT_IMAGE = 560;
const tokensPerImage =
imageInputTokensPerImage ?? LEGACY_TOKENS_PER_INPUT_IMAGE;
Expand All @@ -365,8 +379,18 @@ export async function calculateCosts(
cachedReadTokens -
separatelyPricedCacheWriteTokens,
);
// For providers whose upstream usage already folds image tokens into
// prompt_tokens (OpenAI/Azure/xAI on image-output models), subtract them
// so the image portion isn't billed at both inputPrice and imageInputPrice.
const promptIncludesImageTokens =
isImageOutputModel &&
(provider === "openai" || provider === "azure" || provider === "xai");
const billableTextPromptTokens =
promptIncludesImageTokens && imageInputTokens
? Math.max(0, uncachedPromptTokens - imageInputTokens)
: uncachedPromptTokens;
// inputCost includes both text and image input costs when applicable
const inputCost = new Decimal(uncachedPromptTokens)
const inputCost = new Decimal(billableTextPromptTokens)
.times(inputPrice)
.times(discountMultiplier)
.plus(imageInputCost ?? 0);
Expand Down Expand Up @@ -396,11 +420,15 @@ export async function calculateCosts(
if (imageOutputPricePerToken && outputImageCount > 0) {
const LEGACY_DEFAULT_TOKENS_PER_IMAGE = 1120;
imageOutputTokens =
imageOutputTokensPerImage !== undefined
? outputImageCount * imageOutputTokensPerImage
: totalOutputTokens > 0
? totalOutputTokens
: outputImageCount * LEGACY_DEFAULT_TOKENS_PER_IMAGE;
isImageOutputModel &&
reportedImageOutputTokens &&
reportedImageOutputTokens > 0
? reportedImageOutputTokens
: imageOutputTokensPerImage !== undefined
? outputImageCount * imageOutputTokensPerImage
: totalOutputTokens > 0
? totalOutputTokens
: outputImageCount * LEGACY_DEFAULT_TOKENS_PER_IMAGE;
Comment on lines 445 to +454
const textTokens = Math.max(0, totalOutputTokens - imageOutputTokens);

imageOutputCost = new Decimal(imageOutputTokens)
Expand Down
6 changes: 4 additions & 2 deletions packages/models/src/models/openai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1953,9 +1953,10 @@ export const openaiModels = [
test: "skip",
providerId: "openai",
modelName: "gpt-image-2",
inputPrice: 8 / 1e6,
inputPrice: 5 / 1e6,
outputPrice: 15 / 1e6,
cachedInputPrice: 2 / 1e6,
imageInputPrice: 8 / 1e6,
imageOutputPrice: 30 / 1e6,
requestPrice: 0,
contextSize: 272000,
Expand All @@ -1970,9 +1971,10 @@ export const openaiModels = [
test: "skip",
providerId: "azure",
modelName: "gpt-image-2",
inputPrice: 8 / 1e6,
inputPrice: 5 / 1e6,
outputPrice: 15 / 1e6,
cachedInputPrice: 2 / 1e6,
imageInputPrice: 8 / 1e6,
imageOutputPrice: 30 / 1e6,
discount: 0.2,
requestPrice: 0,
Expand Down
Loading