Skip to content

Add gemma4 unit tests - #2151

Merged
apsonawane merged 12 commits into
mainfrom
asonawane/gemma-tests
May 21, 2026
Merged

Add gemma4 unit tests#2151
apsonawane merged 12 commits into
mainfrom
asonawane/gemma-tests

Conversation

@apsonawane

Copy link
Copy Markdown
Contributor

This pull request adds a comprehensive test suite for the Gemma4 multimodal model, covering model loading, text-only and vision (image) processing, and validation of ONNX model input/output interfaces. These tests ensure that the Gemma4 model and its components function as expected for both text and image modalities.

The most important changes include:

New test coverage for Gemma4 multimodal model:

  • Added unit tests for loading the Gemma4 model and verifying basic text-only and image processing functionality using the onnxruntime_genai library (test_gemma4_models.py).
  • Added parameterized tests for image input, including loading images from file paths and bytes, as well as handling multiple images in a single prompt.
  • Verified that the multimodal processor correctly generates required input fields such as input_ids, pixel_values, and token_type_ids.

ONNX model interface validation:

  • Added tests to validate the required inputs and outputs for the vision, embedding, and text/decoder ONNX models, ensuring compatibility and correct wiring of model components.
  • Included checks for input/output names, data types, and dynamic shapes, particularly for the vision model’s pixel_values input.

Copilot AI review requested due to automatic review settings May 8, 2026 21:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Python pytest module intended to validate Gemma4 multimodal integration in onnxruntime_genai, including basic processor behavior (text-only and vision) and ONNX I/O interface checks for the vision/embedding/decoder components.

Changes:

  • Introduces test/python/test_gemma4_models.py with Gemma4 model load + multimodal processor tests.
  • Adds image-path, image-bytes, and multi-image prompt preprocessing coverage for Gemma4.
  • Adds ONNX graph interface validation assertions for Gemma4 dummy vision/embedding/text models.

Comment thread test/python/test_gemma4_models.py Outdated
Comment thread test/python/test_gemma4_models.py Outdated
Comment thread test/python/test_gemma4_models.py Outdated
Comment thread test/python/test_gemma4_models.py Outdated

@baijumeswani baijumeswani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these tests only run locally or was the intention to also enable it in the CI pipeline?

Comment thread test/python/test_gemma4_models.py
baijumeswani
baijumeswani previously approved these changes May 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 10 changed files in this pull request and generated 1 comment.

Comment thread test/python/test_gemma4_models.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 13 changed files in this pull request and generated 3 comments.

Comment thread test/python/test_gemma4_models.py
Comment thread test/python/test_gemma4_models.py
Comment thread test/python/test_gemma4_models.py
@apsonawane
apsonawane merged commit 2078bfb into main May 21, 2026
16 of 18 checks passed
@apsonawane
apsonawane deleted the asonawane/gemma-tests branch May 21, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants