update MEAI image generation support and basic usage tracking for streaming

[MathiasZander]

Summary

• Added support for image generation configuration (aspect ratio) for models like gemini-2.5-flash-image
• Basic usage information is now included in streaming chat response updates for token tracking

Note

This provides basic usage support for streaming responses. Unlike other MEAI implementations:

• Thinking tokens are not included in output token counts
• AdditionalCounts is not yet implemented

Tested with Vertex AI.

Summary by CodeRabbit

Release Notes

• New Features
  • Image generation is now configurable with customizable aspect ratio settings
  • Response modalities can now be specified in configuration options

[coderabbitai]

Walkthrough

This pull request adds image generation configuration support to the GenerativeAI.Microsoft SDK by introducing response modalities and image aspect ratio properties. Two new constant keys are added to map these properties, new configuration classes are defined in the core generation config, and extension methods are updated to handle the additional properties.

Changes

Cohort / File(s)	Summary
Configuration Constants src/GenerativeAI.Microsoft/AdditionalPropertyKeys.cs	Added two new public constants: ResponseModalities and ImageConfigAspectRatio for mapping additional properties in options.
Core Configuration Models src/GenerativeAI/Types/ContentGeneration/Config/GenerationConfig.cs	Added ImageConfig property to GenerationConfig class and introduced new ImageConfig class with AspectRatio string property for image generation settings.
Extension Mapping Logic src/GenerativeAI.Microsoft/Extensions/MicrosoftExtensions.cs	Updated ToGenerationConfig to map ResponseModalities and ImageConfigAspectRatio from additional properties; refactored ToChatResponseUpdate to pre-build contents collection and properly include UsageContent when available.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• Update to M.E.AI 9.3.0-preview.1.25161.3 #25: Both PRs modify MicrosoftExtensions.cs and update the ToChatResponseUpdate method to reconstruct how response contents are aggregated and assembled.

Poem

🐰 A rabbit hops through configs new,
With modalities and ratios true,
Images dream in aspect's frame,
Extensions mapped—no more the same! 🖼️✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The pull request title "update MEAI image generation support and basic usage tracking for streaming" directly aligns with the actual changes made in the changeset. The title accurately captures the two primary objectives: adding ImageConfig support with AspectRatio for image generation (reflected in the new ImageConfig class and constants) and enhancing usage tracking in streaming responses (visible in the improved ToChatResponseUpdate implementation). The title is specific and concrete, avoiding vague terminology while remaining concise and clear enough for reviewers to understand the main purpose of the changes.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

src/GenerativeAI.Microsoft/Extensions/MicrosoftExtensions.cs (1)

345-349: Consider adding aspect ratio format validation.

The implementation correctly instantiates ImageConfig and sets the aspect ratio. However, there's no validation for the aspect ratio format. Consider validating against expected patterns like "16:9", "9:16", "1:1", "4:3", "3:4" to catch configuration errors early.

Example validation:

 if (options.AdditionalProperties.TryGetValue(AdditionalPropertiesKeys.ImageConfigAspectRatio, out string? aspectRatio))
 {
+    // Validate aspect ratio format (optional but recommended)
+    if (!string.IsNullOrEmpty(aspectRatio) && !System.Text.RegularExpressions.Regex.IsMatch(aspectRatio, @"^\d+:\d+$"))
+    {
+        throw new ArgumentException($"Invalid aspect ratio format: {aspectRatio}. Expected format: 'width:height' (e.g., '16:9')");
+    }
     config.ImageConfig ??= new ImageConfig();
     config.ImageConfig.AspectRatio = aspectRatio;
 }

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5f87cba and 6daaf7e.

📒 Files selected for processing (3)

• src/GenerativeAI.Microsoft/AdditionalPropertyKeys.cs (1 hunks)
• src/GenerativeAI.Microsoft/Extensions/MicrosoftExtensions.cs (2 hunks)
• src/GenerativeAI/Types/ContentGeneration/Config/GenerationConfig.cs (2 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

src/GenerativeAI.Microsoft/Extensions/MicrosoftExtensions.cs (4)

src/GenerativeAI.Microsoft/AdditionalPropertyKeys.cs (1)

• AdditionalPropertiesKeys (6-27)

src/GenerativeAI/Types/ContentGeneration/Config/GenerationConfig.cs (1)

• ImageConfig (335-342)

src/GenerativeAI/Types/ContentGeneration/Inputs/Content.cs (4)

• Content (11-58)
• Content (16-19)
• Content (25-29)
• Content (40-44)

src/GenerativeAI/Types/ContentGeneration/Inputs/Part.cs (3)

• Part (12-96)
• Part (69-72)
• Part (79-82)

🔇 Additional comments (6)

src/GenerativeAI.Microsoft/AdditionalPropertyKeys.cs (1)

18-26: LGTM! Clean constant additions for image generation support.

The new constants are well-documented and integrate properly with the generation config. The key name "AspectRatio" is generic but acceptable given it's namespaced within AdditionalPropertiesKeys.ImageConfig*.

src/GenerativeAI.Microsoft/Extensions/MicrosoftExtensions.cs (3)

340-343: LGTM! Response modalities properly configured.

The implementation correctly reads the modalities list from additional properties and assigns it to the generation config.

---

408-419: Good refactoring for streaming usage tracking.

The change to pre-build the contents collection and append UsageContent when usage data is available is well-implemented and enables the framework to aggregate usage information during streaming. This aligns with the PR objectives for basic usage tracking.

---

546-556: Review verified: Implementation is correct.

The Vertex AI API's output token count does not include any thinking tokens; those are provided in a separate field. The implementation correctly uses CandidatesTokenCount directly for OutputTokenCount, which already excludes thinking tokens as required by the PR objectives. The separate ThoughtsTokenCount field tracks thinking tokens independently, so no additional filtering is needed here.

src/GenerativeAI/Types/ContentGeneration/Config/GenerationConfig.cs (2)

213-217: LGTM! ImageConfig property properly integrated.

The new property follows the existing patterns in GenerationConfig with appropriate JSON serialization attributes and clear documentation.

---

332-342: LGTM! Well-defined ImageConfig class.

The ImageConfig class is cleanly implemented with:

• Clear documentation including common aspect ratio examples
• Proper JSON property naming convention (aspectRatio)
• Nullable string type for optional configuration