test(integration): cover images endpoint contracts

[devin-ai-integration]

TLDR

Problem this solves:

• /v1/images/edits alias spellings and the /images/* and Azure path aliases had no proxy-level wire test
• The image[] and mask[] 400 regression shipped because of that gap
• gpt-image generation bodies were only partially asserted

How it solves it:

• Drive the real proxy with every documented image edit spelling
• Assert every outbound multipart part, file vs text, part by part
• Assert full upstream JSON bodies for gpt-image generation and fal edits

User Flow

Before: a team that edits images through the proxy with the OpenAI SDK could ship a proxy build that broke them, because no test sent the request the way the SDK does

1. They call client.images.edit(image=[a, b], mask=m, prompt=...), which sends POST https://litellm-domain/v1/images/edits with image[] and mask[] parts
2. A proxy change starts forwarding a stringified upload next to the real file and the test suite stays green
3. OpenAI answers 400 "expected a file, but got a string" and every edit fails

After: the same change fails the integration suite before it can merge

1. They send the same POST https://litellm-domain/v1/images/edits with image[] and mask[] parts
2. The upstream receives exactly one file part per upload in upload order and one mask file part, with no text part carrying an upload
3. They get the edited image back with a 200

Relevant issues

Follow-up to #39512, which fixed the image[] / mask[] regression introduced by #38104

Linear ticket

Related to LIT-6816, LIT-7940 and LIT-6807

What is now covered

Every inbound spelling of /v1/images/edits now goes through the real proxy against a scripted OpenAI upstream, and the upstream handler parses the multipart body into an ordered list of parts instead of the old name-to-value dicts, so a duplicate or stringified part can no longer hide. The covered spellings are image, image[], repeated image[], repeated image, mask with image, mask[] with image[], and the real openai-python images.edit(image=[a, b], mask=m, n=2) call. Each test asserts the exact text parts (sorted, duplicates kept), every file part with its name, filename, content type and bytes equal to the upload, the client-parsed response, and that exactly one upstream request happened. Sending both image and image[], or both mask and mask[], now has a test asserting the 422 body and zero upstream requests. A multipart n=2 is asserted to reach fal as the integer num_images: 2 in the full JSON body, and to reach OpenAI as one text part n=2. The OpenAI SDK's images.generate call with every gpt-image field is asserted against the full upstream JSON body, with n as an integer, and the SDK-parsed data and usage

The bracketed image[] test is parametrized over /v1/images/edits, /images/edits and /openai/deployments/{model}/images/edits, and the full gpt-image generation test drives the SDK at /v1/images/generations, /images/generations and, through AzureOpenAI, /openai/deployments/{model}/images/generations, so deleting any one route decorator fails a test. Two mask[] parts are asserted to reach OpenAI as exactly one mask file part carrying the first upload's bytes, which is the contract OpenAIImageEditConfig implements today

/v1/images/variations is a product bug on main: litellm.image_variation exists but the proxy registers no route, so POST /v1/images/variations returns 404 {"detail":"Not Found"}. tests/integration/providers/test_openai_image_variation_wire.py::test_image_variation_multipart_image_and_string_n_reach_openai_as_file_part holds the full contract (one image file part with the upload bytes, text parts model and n=2, the parsed response, one upstream request) behind pytest.skip("BUG: POST /v1/images/variations is not routed by the proxy"). With the skip removed it fails on response.status_code == 200 with 404

Gap table

Every test was proven by mutation. The one-line change below was applied under litellm/, the proxy restarted, the test run and seen failing on the named assertion, then the change reverted and the test seen passing

Gap	Test node	Mutation	Assertion it tripped
images-1	tests/integration/providers/test_openai_image_edit_wire.py::test_image_edit_bracketed_image_field_reaches_openai_as_one_file_part[v1,unversioned,azure_deployment]	endpoints.py image_edit_api: if key not in BRACKETED_FILE_FIELDS to if True	all three ids failed on _text_parts(parts) == (("model", ...), ("prompt", ...), ("seed", "42")) with an extra ("image[]", "UploadFile(filename='red_circle.png', ...)")
edit path alias	...test_image_edit_bracketed_image_field_reaches_openai_as_one_file_part[unversioned]	endpoints.py: delete the @router.post("/images/edits", ...) decorator	response.status_code == 200 got 404 {"detail":"Not Found"}
edit path alias	...test_image_edit_bracketed_image_field_reaches_openai_as_one_file_part[azure_deployment]	endpoints.py: delete the @router.post("/openai/deployments/{model:path}/images/edits", ...) decorator	response.status_code == 200 got 404
repeated mask[]	...test_openai_image_edit_wire.py::test_image_edit_repeated_bracketed_mask_parts_reach_openai_as_first_mask_only	litellm/llms/openai/image_edit/transformation.py: _mask = _mask[0] if _mask else None to _mask = _mask[-1] if _mask else None	_file_parts(parts) == (image, first_mask) got the second mask's bytes
images-2	...test_openai_image_edit_wire.py::test_image_edit_repeated_bracketed_image_parts_reach_openai_in_upload_order	endpoints.py batch_to_bytesio: for u in uploads to for u in uploads[:1] (also fails under the images-1 mutation on _text_parts)	_file_parts(parts) == (first, second) got one file part
images-3	...test_openai_image_edit_wire.py::test_image_edit_bracketed_mask_reaches_openai_as_mask_file_part	endpoints.py: BRACKETED_FILE_FIELDS = frozenset({IMAGE_ARRAY_FIELD, MASK_ARRAY_FIELD}) to frozenset({IMAGE_ARRAY_FIELD}) (also fails under images-1)	_text_parts(parts) == (("model", ...), ("prompt", ...)) got an extra ("mask[]", "UploadFile(filename=...)")
images-4	...test_openai_image_edit_wire.py::test_image_edit_plain_mask_reaches_openai_as_mask_file_part	endpoints.py: mask_files = await batch_to_bytesio(mask) to mask_files = None	parsed.is_multipart() in _multipart_parts failed, the upstream got no multipart body
images-5	...test_openai_image_edit_wire.py::test_image_edit_repeated_plain_image_parts_reach_openai_as_two_file_parts	endpoints.py: batch_to_bytesio(image) to batch_to_bytesio(image[-1:] if image else image)	_file_parts(parts) == (first, second) got only the second upload
images-6	...test_openai_image_edit_wire.py::test_image_edit_rejects_plain_and_bracketed_spelling_together_without_calling_upstream[image]	endpoints.py: delete the image / image[] conflict guard (two lines)	response.status_code == 422 got 200
images-6	...test_openai_image_edit_wire.py::test_image_edit_rejects_plain_and_bracketed_spelling_together_without_calling_upstream[mask]	endpoints.py: delete the mask / mask[] conflict guard (two lines)	response.status_code == 422 got 200
images-7	tests/integration/providers/test_fal_ai_image_wire.py::test_fal_gpt_image_25_edit_sends_multipart_n_as_integer_num_images	endpoints.py: numeric_fields=IMAGE_EDIT_NUMERIC_FORM_FIELDS to numeric_fields={}	upstream body == {..., "num_images": 2} got "num_images": "2"
images-8	tests/integration/providers/test_image_gen_drop_params_wire.py::test_gpt_image_generation_forwards_every_sdk_field_once_with_integer_n[v1,unversioned,azure_deployment]	litellm/llms/openai/openai.py image_generation: **optional_params to **{k: v for k, v in optional_params.items() if k != "n"}	all three ids failed on upstream body == {..., "n": 2, ...} with no n
generation path alias	...test_gpt_image_generation_forwards_every_sdk_field_once_with_integer_n[unversioned]	endpoints.py: delete the @router.post("/images/generations", ...) decorator	client.images.generate(...) raised openai.NotFoundError 404
generation path alias	...test_gpt_image_generation_forwards_every_sdk_field_once_with_integer_n[azure_deployment]	endpoints.py: delete the @router.post("/openai/deployments/{model:path}/images/generations", ...) decorator	client.images.generate(...) raised openai.NotFoundError 404
variations (BUG)	tests/integration/providers/test_openai_image_variation_wire.py::test_image_variation_multipart_image_and_string_n_reach_openai_as_file_part	none, the route is absent on main, so the skip removed is the red run	response.status_code == 200 got 404 {"detail":"Not Found"}
images-1, images-2, images-4 (SDK spelling)	...test_openai_image_edit_wire.py::test_openai_sdk_image_edit_with_image_list_and_mask_reaches_openai_as_file_parts	endpoints.py image_edit_api: if key not in BRACKETED_FILE_FIELDS to if True	_text_parts(parts) == (("model", ...), ("n", "2"), ("prompt", ...)) got an extra ("image[]", "UploadFile(filename='second.png', ...)")

The OpenAI half of images-7 (text part n=2 exactly once) is asserted inside the images-2 test. It stays green under the images-7 mutation because OpenAI receives the string form either way, so the integer contract is proven only on the fal side. The requested second mutation, making the multipart flattener str() an upload, is already the live behavior on main (_form_field_value in litellm/litellm_core_utils/llm_request_utils.py has no file branch), so it is not a separate edit. The raw-alias filter is the only guard, and reverting it reproduces the full RCA symptom on top of the live str() fallback: text parts 'image[]'='UploadFile(...)' and 'mask[]'='UploadFile(...)' reach the upstream, which the images-1 and images-3 tests reject part by part

Screenshots / Proof of Fix

Tests-only change, so the proof is the integration selection run against the CircleCI-shaped rig

Rig

Postgres on 127.0.0.1:5432 (circle_test), Redis on 127.0.0.1:6379, the owned upstream on 127.0.0.1:8190, and the proxy started the way .circleci/scripts/run_integration.sh does it

python -m integration._support.proxy --config tests/integration/proxy_config.yaml \
  --host 127.0.0.1 --port 4000 --num_workers 2 --telemetry False --use_prisma_db_push

After (ef4ca2e)

1. uv run --no-sync python tests/integration/run.py providers tests/integration/providers/test_openai_image_edit_wire.py tests/integration/providers/test_fal_ai_image_wire.py tests/integration/providers/test_image_gen_drop_params_wire.py tests/integration/providers/test_openai_image_variation_wire.py gave 24 passed, 1 skipped in 33.97s, the skip being the variations BUG test
2. The same command again gave 24 passed, 1 skipped in 29.85s with the identical 25 selected node ids
3. make check gave check: PASS

Type

Test

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem

Caveats (if any)

Low

• /v1/images/variations has no proxy route on main, so its test is skipped as a product bug until the route lands
• Outbound filenames are asserted as image.png / mask.png, which the provider transformation sets

The diff changes only test functions and helpers, and no product symbol, so the /live-pr-risk dependency graph is empty and there is no base vs head A/B to run

• ef4ca2e passes /live-pr-risk

Link to Devin session: https://app.devin.ai/sessions/bd64a051f8c54192af4b2e29eebbf7ee
Open in Devin Desktop: https://app.devin.ai/desktop/session/bd64a051f8c54192af4b2e29eebbf7ee?variant=devin
Requested by: @mateo-berri

---

Note

Low Risk
Test-only changes with no production code; CI runtime may increase slightly from new integration cases.

Overview
Adds integration wire tests so the proxy’s image APIs are exercised the way clients (OpenAI SDK, multipart uploads, path aliases) actually call them, with assertions on the exact upstream request bodies and multipart parts.

Image edits coverage is expanded heavily: image / image[], repeated uploads, mask / mask[] (including first-mask-only for duplicate mask[]), OpenAI SDK images.edit with image list + mask, route aliases (/v1, /images, Azure deployment path), and 422 when image and image[] (or mask and mask[]) are mixed—with no upstream call. Multipart helpers now assert ordered file parts (name, filename, type, bytes) instead of loose dicts.

GPT image generation gets a parametrized test that drives the official SDK across the same path aliases and checks the full JSON upstream payload (including n as integer and usage parsing). Fal edit gets a test that string form n=2 becomes num_images: 2 in JSON.

A new variations wire test documents the desired contract but is pytest.skip’d because POST /v1/images/variations is not routed on main (404).

^{Reviewed by Cursor Bugbot for commit ef4ca2e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Low risk] Adds integration tests for image endpoint contracts.

The test-only PR appears safe to merge.

Summary

The PR adds proxy-level image endpoint contract tests.

• Checks edit aliases, multipart file parts, upload order, and an OpenAI SDK edit request.
• Checks generation aliases and outbound image-generation fields.
• Documents the currently unavailable variation route with a skipped test.

_{Reviews (3) · Last reviewed commit: "test(integration): cover image route ali..."}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ef4ca2e. Configure here.}