fix: replicate harness test and gemini extra param - #5527
Merged
akshaydeo merged 1 commit intoJul 24, 2026
Merged
Conversation
|
tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Contributor
📝 WalkthroughWalkthroughGemini media resolution now survives conversion through Bifrost response parameters. Replicate input image mapping gains broader regression coverage, provider-harness validation, and provider filtering keywords. ChangesGemini media resolution
Replicate input image mapping
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
akshaydeo
approved these changes
Jul 24, 2026
Contributor
Merge activity
|
akshaydeo
deleted the
07-24-fix_replicate_harness_test_and_gemini_extra_param
branch
July 24, 2026 12:47
akshaydeo
pushed a commit
that referenced
this pull request
Jul 24, 2026
## Summary
Two silent data-loss bugs are fixed: `generationConfig.mediaResolution` was dropped during the Gemini → Bifrost → Gemini round-trip on the GenAI `generateContent` path, causing image token counts to ignore the field entirely; and `input_images` was dropped on the Replicate `/v1/images/generations` path because the model-specific field mapping (`image_prompt` / `input_image` / `image` / `input_images`) was only applied on the edits path, so image-to-image requests silently generated from the prompt alone.
## Changes
- **Gemini `mediaResolution` round-trip fix**: `convertGenerationConfigToResponsesParameters` now writes `config.MediaResolution` into `ExtraParams["media_resolution"]` on the inbound leg, and `convertParamsToGenerationConfigResponses` reads it back out on the outbound leg. Previously neither leg handled the field, so `MEDIA_RESOLUTION_LOW`/`HIGH` had no effect on prompt token billing.
- **Replicate `input_images` on generations path**: The model-specific field mapping that routes `input_images` to the correct per-model field (`input_image` for kontext models, `image_prompt` for flux-pro variants, `image` for flux-dev/fill/krea, `input_images` array for unknown models) is now applied on the `/v1/images/generations` path in addition to the edits path.
- **Gemini test**: `TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip` verifies that `MEDIA_RESOLUTION_LOW` survives a full Gemini → Bifrost → Gemini round-trip via `ExtraParams`.
- **Replicate tests**: The three flat `InputImages_*` table cases are replaced by `TestReplicateImageGenerationInputImagesFieldMapping`, a structured test that covers every model in the mapping table, the default fallback, version-suffix stripping, case-insensitive model name matching, raw base64 → data URL normalization, invalid URL rejection, and the no-input-images baseline.
- **E2E harness**: Two new test groups are added. Group 30 sends a real landscape image to `flux-kontext-pro` via `/v1/images/generations` with `aspect_ratio: match_input_image` and asserts the output preserves the landscape aspect ratio (a square output proves the image was dropped). Group 31 sends the same 1024×1024 JPEG with `MEDIA_RESOLUTION_LOW` and `MEDIA_RESOLUTION_HIGH` to both the GenAI and Vertex routes and asserts the HIGH request reports more prompt tokens than LOW (equal counts prove the field was dropped).
- **E2E filter**: `replicate` is added as a named provider keyword so the collection runner can target Replicate-specific cases in isolation.
## Type of change
- [x] Bug fix
## Affected areas
- [x] Core (Go)
- [x] Providers/Integrations
## How to test
```sh
# Unit tests
go test ./core/providers/gemini/... -run TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip -v
go test ./core/providers/replicate/... -run TestReplicateImageGenerationInputImagesFieldMapping -v
# Full suite
go test ./...
```
For E2E validation, run the provider harness collection against a live environment with `{{genaiKey}}` and a Replicate-enabled `{{baseUrl}}`. Group 30 requires a Replicate API key; group 31 requires a Gemini/GenAI key. Group 31.3 and 31.4 additionally require Vertex credentials and will skip cleanly if not configured.
## Breaking changes
- [x] No
## Related issues
Fixes silent `mediaResolution` drop on Gemini GenAI path. Fixes silent `input_images` drop on Replicate image generations path.
## Security considerations
No new auth surfaces, secrets, or PII handling introduced. The E2E fixture image is a public JPEG from Google's generative AI downloads bucket.
## Checklist
- [x] I added/updated tests where appropriate
- [x] I verified builds succeed (Go and UI)
9 tasks
akhsaul
pushed a commit
to akhsaul/bifrost
that referenced
this pull request
Aug 27, 2026
## Summary
Two silent data-loss bugs are fixed: `generationConfig.mediaResolution` was dropped during the Gemini → Bifrost → Gemini round-trip on the GenAI `generateContent` path, causing image token counts to ignore the field entirely; and `input_images` was dropped on the Replicate `/v1/images/generations` path because the model-specific field mapping (`image_prompt` / `input_image` / `image` / `input_images`) was only applied on the edits path, so image-to-image requests silently generated from the prompt alone.
## Changes
- **Gemini `mediaResolution` round-trip fix**: `convertGenerationConfigToResponsesParameters` now writes `config.MediaResolution` into `ExtraParams["media_resolution"]` on the inbound leg, and `convertParamsToGenerationConfigResponses` reads it back out on the outbound leg. Previously neither leg handled the field, so `MEDIA_RESOLUTION_LOW`/`HIGH` had no effect on prompt token billing.
- **Replicate `input_images` on generations path**: The model-specific field mapping that routes `input_images` to the correct per-model field (`input_image` for kontext models, `image_prompt` for flux-pro variants, `image` for flux-dev/fill/krea, `input_images` array for unknown models) is now applied on the `/v1/images/generations` path in addition to the edits path.
- **Gemini test**: `TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip` verifies that `MEDIA_RESOLUTION_LOW` survives a full Gemini → Bifrost → Gemini round-trip via `ExtraParams`.
- **Replicate tests**: The three flat `InputImages_*` table cases are replaced by `TestReplicateImageGenerationInputImagesFieldMapping`, a structured test that covers every model in the mapping table, the default fallback, version-suffix stripping, case-insensitive model name matching, raw base64 → data URL normalization, invalid URL rejection, and the no-input-images baseline.
- **E2E harness**: Two new test groups are added. Group 30 sends a real landscape image to `flux-kontext-pro` via `/v1/images/generations` with `aspect_ratio: match_input_image` and asserts the output preserves the landscape aspect ratio (a square output proves the image was dropped). Group 31 sends the same 1024×1024 JPEG with `MEDIA_RESOLUTION_LOW` and `MEDIA_RESOLUTION_HIGH` to both the GenAI and Vertex routes and asserts the HIGH request reports more prompt tokens than LOW (equal counts prove the field was dropped).
- **E2E filter**: `replicate` is added as a named provider keyword so the collection runner can target Replicate-specific cases in isolation.
## Type of change
- [x] Bug fix
## Affected areas
- [x] Core (Go)
- [x] Providers/Integrations
## How to test
```sh
# Unit tests
go test ./core/providers/gemini/... -run TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip -v
go test ./core/providers/replicate/... -run TestReplicateImageGenerationInputImagesFieldMapping -v
# Full suite
go test ./...
```
For E2E validation, run the provider harness collection against a live environment with `{{genaiKey}}` and a Replicate-enabled `{{baseUrl}}`. Group 30 requires a Replicate API key; group 31 requires a Gemini/GenAI key. Group 31.3 and 31.4 additionally require Vertex credentials and will skip cleanly if not configured.
## Breaking changes
- [x] No
## Related issues
Fixes silent `mediaResolution` drop on Gemini GenAI path. Fixes silent `input_images` drop on Replicate image generations path.
## Security considerations
No new auth surfaces, secrets, or PII handling introduced. The E2E fixture image is a public JPEG from Google's generative AI downloads bucket.
## Checklist
- [x] I added/updated tests where appropriate
- [x] I verified builds succeed (Go and UI)
occcat
pushed a commit
to occcat/bifrost
that referenced
this pull request
Sep 2, 2026
## Summary
Two silent data-loss bugs are fixed: `generationConfig.mediaResolution` was dropped during the Gemini → Bifrost → Gemini round-trip on the GenAI `generateContent` path, causing image token counts to ignore the field entirely; and `input_images` was dropped on the Replicate `/v1/images/generations` path because the model-specific field mapping (`image_prompt` / `input_image` / `image` / `input_images`) was only applied on the edits path, so image-to-image requests silently generated from the prompt alone.
## Changes
- **Gemini `mediaResolution` round-trip fix**: `convertGenerationConfigToResponsesParameters` now writes `config.MediaResolution` into `ExtraParams["media_resolution"]` on the inbound leg, and `convertParamsToGenerationConfigResponses` reads it back out on the outbound leg. Previously neither leg handled the field, so `MEDIA_RESOLUTION_LOW`/`HIGH` had no effect on prompt token billing.
- **Replicate `input_images` on generations path**: The model-specific field mapping that routes `input_images` to the correct per-model field (`input_image` for kontext models, `image_prompt` for flux-pro variants, `image` for flux-dev/fill/krea, `input_images` array for unknown models) is now applied on the `/v1/images/generations` path in addition to the edits path.
- **Gemini test**: `TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip` verifies that `MEDIA_RESOLUTION_LOW` survives a full Gemini → Bifrost → Gemini round-trip via `ExtraParams`.
- **Replicate tests**: The three flat `InputImages_*` table cases are replaced by `TestReplicateImageGenerationInputImagesFieldMapping`, a structured test that covers every model in the mapping table, the default fallback, version-suffix stripping, case-insensitive model name matching, raw base64 → data URL normalization, invalid URL rejection, and the no-input-images baseline.
- **E2E harness**: Two new test groups are added. Group 30 sends a real landscape image to `flux-kontext-pro` via `/v1/images/generations` with `aspect_ratio: match_input_image` and asserts the output preserves the landscape aspect ratio (a square output proves the image was dropped). Group 31 sends the same 1024×1024 JPEG with `MEDIA_RESOLUTION_LOW` and `MEDIA_RESOLUTION_HIGH` to both the GenAI and Vertex routes and asserts the HIGH request reports more prompt tokens than LOW (equal counts prove the field was dropped).
- **E2E filter**: `replicate` is added as a named provider keyword so the collection runner can target Replicate-specific cases in isolation.
## Type of change
- [x] Bug fix
## Affected areas
- [x] Core (Go)
- [x] Providers/Integrations
## How to test
```sh
# Unit tests
go test ./core/providers/gemini/... -run TestGenAIMediaResolution_PreservedThroughBifrostRoundTrip -v
go test ./core/providers/replicate/... -run TestReplicateImageGenerationInputImagesFieldMapping -v
# Full suite
go test ./...
```
For E2E validation, run the provider harness collection against a live environment with `{{genaiKey}}` and a Replicate-enabled `{{baseUrl}}`. Group 30 requires a Replicate API key; group 31 requires a Gemini/GenAI key. Group 31.3 and 31.4 additionally require Vertex credentials and will skip cleanly if not configured.
## Breaking changes
- [x] No
## Related issues
Fixes silent `mediaResolution` drop on Gemini GenAI path. Fixes silent `input_images` drop on Replicate image generations path.
## Security considerations
No new auth surfaces, secrets, or PII handling introduced. The E2E fixture image is a public JPEG from Google's generative AI downloads bucket.
## Checklist
- [x] I added/updated tests where appropriate
- [x] I verified builds succeed (Go and UI)
This was referenced Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two silent data-loss bugs are fixed:
generationConfig.mediaResolutionwas dropped during the Gemini → Bifrost → Gemini round-trip on the GenAIgenerateContentpath, causing image token counts to ignore the field entirely; andinput_imageswas dropped on the Replicate/v1/images/generationspath because the model-specific field mapping (image_prompt/input_image/image/input_images) was only applied on the edits path, so image-to-image requests silently generated from the prompt alone.Changes
mediaResolutionround-trip fix:convertGenerationConfigToResponsesParametersnow writesconfig.MediaResolutionintoExtraParams["media_resolution"]on the inbound leg, andconvertParamsToGenerationConfigResponsesreads it back out on the outbound leg. Previously neither leg handled the field, soMEDIA_RESOLUTION_LOW/HIGHhad no effect on prompt token billing.input_imageson generations path: The model-specific field mapping that routesinput_imagesto the correct per-model field (input_imagefor kontext models,image_promptfor flux-pro variants,imagefor flux-dev/fill/krea,input_imagesarray for unknown models) is now applied on the/v1/images/generationspath in addition to the edits path.TestGenAIMediaResolution_PreservedThroughBifrostRoundTripverifies thatMEDIA_RESOLUTION_LOWsurvives a full Gemini → Bifrost → Gemini round-trip viaExtraParams.InputImages_*table cases are replaced byTestReplicateImageGenerationInputImagesFieldMapping, a structured test that covers every model in the mapping table, the default fallback, version-suffix stripping, case-insensitive model name matching, raw base64 → data URL normalization, invalid URL rejection, and the no-input-images baseline.flux-kontext-provia/v1/images/generationswithaspect_ratio: match_input_imageand asserts the output preserves the landscape aspect ratio (a square output proves the image was dropped). Group 31 sends the same 1024×1024 JPEG withMEDIA_RESOLUTION_LOWandMEDIA_RESOLUTION_HIGHto both the GenAI and Vertex routes and asserts the HIGH request reports more prompt tokens than LOW (equal counts prove the field was dropped).replicateis added as a named provider keyword so the collection runner can target Replicate-specific cases in isolation.Type of change
Affected areas
How to test
For E2E validation, run the provider harness collection against a live environment with
{{genaiKey}}and a Replicate-enabled{{baseUrl}}. Group 30 requires a Replicate API key; group 31 requires a Gemini/GenAI key. Group 31.3 and 31.4 additionally require Vertex credentials and will skip cleanly if not configured.Breaking changes
Related issues
Fixes silent
mediaResolutiondrop on Gemini GenAI path. Fixes silentinput_imagesdrop on Replicate image generations path.Security considerations
No new auth surfaces, secrets, or PII handling introduced. The E2E fixture image is a public JPEG from Google's generative AI downloads bucket.
Checklist