fix(images): label transcoded image parts with their real media type on the OpenAI wire

[murdore]

Summary

Resolves the review finding left open on the merged #1738: with imageOptions.outputFormat: "jpeg", real JPEG bytes reached the OpenAI chat-completions wire inside a data URI labelled image/png.

Two independent drops caused it, and each one alone reproduces the bug:

• src/lib/core/modules/MessageBuilder.ts rebuilt every image content part from type and image only, discarding the resolved media type in both buildMessages() and buildMessagesForStream().
• src/lib/providers/openaiChatCompletionsClient.ts: convertContentForOpenAI never passed the part's media type to imageDataToURL, so every raw-base64 image fell back to its hardcoded image/png default.

Tests

test/continuous-test-suite-vision-image-format.ts now asserts the wire label as well as the decoded bytes, on both generate() and stream(). It is offline (loopback mock chat server), so it runs without credentials.

• 5/5 pass on a fresh build.
• stream() serializes to the wire at its own call site (doStream), so it gets its own label test. Stripping the media type at only that site now fails exactly that test with ✗ and exit 1, while every generate() test stays green. Before this change the same mutant passed all four tests.
• Mutation runs during review showed each of the two source edits is needed on its own.
• pnpm run check 0 errors, pnpm run lint 0 errors, docs:api regenerated with no diff (neither file is in the public typedoc output).

Where the other deferred review findings went

This PR was planned to consolidate several deferred findings. Most turned out to live in code that exists only on still-open PRs, so a release-based PR cannot fix them:

Finding (source PR)	Routed to	Why
OpenAI image MIME label (#1738, merged)	this PR	code is on release
isError severity classification (#1619)	after #1619 merges	the breaker path it must gate exists only on #1619's branch
stdio PID-based SIGKILL fallback (#1683)	dropped	the SDK's StdioClientTransport.close() already escalates SIGTERM→SIGKILL; a PID fallback could signal a recycled PID after auto-restart
LiteLLM/Ollama probe ungated (#1735)	redesign inside #1735	the straightforward gate made auto-select pick litellm on a machine with no proxy running
logger dispose test, local type (#1743)	fixed in #1743	test file exists only on that branch
htmlToMarkdown link escaping, StreamOptions docs (#1755)	fixed in #1755	file and officeOptions type exist only on that branch
search-index byCodepoint, empty-heading anchors (#1763)	fixed in #1763	plugin code exists only on that branch
sync-docs CI wiring, magic number (#1771)	n/a	#1771 is superseded by the merged #1774

Summary by CodeRabbit

• Bug Fixes
  • Images are now sent with their correct format instead of being mislabeled as PNG. This improves compatibility when using JPEG and other supported image formats in both regular and streaming requests.

[github-actions]

✅ Single Commit Policy - COMPLIANT

Status: Policy requirements met • 1 commit • Valid format • Ready for merge

📊 View validation details

📝 Commit Details

• Hash: ab9918d6e451d377e8d84e8395b7e60d21d3fade
• Message: fix(images): label transcoded image parts with their real media type on the OpenAI wire
• Author: Sachin Sharma

✅ Validation Results

• Single commit requirement met
• No merge commits in branch
• Semantic commit message format verified
• Ready for squash merge to release branch

---

🤖 Automated validation by NeuroLink Single Commit Enforcement

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: juspay/neurolink/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: e56f7413-9d18-4d8b-89fe-91f30bc83394

📥 Commits

Reviewing files that changed from the base of the PR and between f536fd0 and ab9918d.

📒 Files selected for processing (3)

• src/lib/core/modules/MessageBuilder.ts
• src/lib/providers/openaiChatCompletionsClient.ts
• test/continuous-test-suite-vision-image-format.ts

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.

---

📝 Walkthrough

Walkthrough

Image conversion now preserves resolved media types in both message-building paths. OpenAI serialization uses the media type when labeling image data URIs. Tests check the JPEG label in generate and stream requests.

Changes

Image media type handling

Layer / File(s)	Summary
Preserve media types in message conversion src/lib/core/modules/MessageBuilder.ts	Both message-building paths include an image item's mediaType or mimeType on the resulting ImagePart when available.
Serialize and verify image media types src/lib/providers/openaiChatCompletionsClient.ts, test/continuous-test-suite-vision-image-format.ts	OpenAI content conversion uses the image part's media type when creating image data URIs. Tests check that generate and stream requests label JPEG data as image/jpeg.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Merge Risk: ⚪ Minimal · up to ab991

The image-label change is mergeable after normal checks; the reported test-skip concern is not reachable in these tests.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: preserving the real media type for transcoded image parts on the OpenAI wire. It is specific and concise enough for the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

Warning

Some tools did not complete. Review the errors below.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

src/lib/core/modules/MessageBuilder.ts

Parsing error: Unable to parse the specified 'tsconfig' file. Ensure it's correct and has valid syntax.

error TS5012: Cannot read file '/.svelte-kit/tsconfig.json': ENOENT: no such file or directory, open '/.svelte-kit/tsconfig.json'.

src/lib/providers/openaiChatCompletionsClient.ts

ESLint skipped: the matched ESLint configuration already failed (missing-dependency).

test/continuous-test-suite-vision-image-format.ts

ESLint skipped: the matched ESLint configuration already failed (missing-dependency).

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🎉 This PR is included in version 12.21.2 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀