test(bedrock): switch image gen live test off EOL Titan to Nova Canvas - #31937
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Greptile SummaryThis PR replaces the EOL
Confidence Score: 4/5The change is a minimal, isolated swap of a model ID in one test; no production code is affected and the test logic is otherwise identical to what it replaced. The renamed test continues to make a real HTTP call to Bedrock inside tests/image_gen_tests/, the same violation the original Titan test had. The rest of the file uses mocked responses for this exact reason. The live call can intermittently fail in environments without AWS credentials and may not run cleanly for contributors who clone the repo locally. tests/image_gen_tests/test_bedrock_image_gen_unit_tests.py — the live-call test at the bottom of the file should either be mocked or relocated to tests/e2e/
|
| Filename | Overview |
|---|---|
| tests/image_gen_tests/test_bedrock_image_gen_unit_tests.py | Renames test_amazon_titan_image_gen to test_amazon_nova_canvas_image_gen and swaps the model ID from the EOL Titan v2 to Nova Canvas v1:0. The test continues to make a live Bedrock network call, which is not permitted in this folder per project rules. |
Comments Outside Diff (1)
-
tests/image_gen_tests/test_bedrock_image_gen_unit_tests.py, line 530-544 (link)Live network call outside the allowed folder
This test calls
litellm.image_generationagainst the real Bedrock endpoint (aws_region_name="us-east-1") without any mocking. The repository rule restricts live network calls totests/e2e/; only mock tests are permitted intests/image_gen_tests/. The previous Titan test had the same issue, and this PR carries that pattern forward into the Nova Canvas replacement. To comply, either mock the Bedrock HTTP response (as other unit tests in this file do) or move the live test totests/e2e/.Rule Used: What: prevent any tests from being added here that... (source)
Reviews (1): Last reviewed commit: "test(bedrock): switch image gen live tes..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
912ca62
into
litellm_internal_staging
BerriAI#31937) Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Relevant issues
CI was failing on
tests/image_gen_tests/test_bedrock_image_gen_unit_tests.py::test_amazon_titan_image_genwithThe test made a live Bedrock call to
amazon.titan-image-generator-v2:0. Per the Amazon Bedrock model lifecycle table, Titan Image Generator G1 v2 has an EOL date of June 30, 2026, so as of July 2026 the call fails permanently in every region. v1 was already retired earlier, which is why the test had previously moved to v2Changes
Point the live image-gen test at
amazon.nova-canvas-v1:0, Amazon's current Bedrock image model, and rename the test totest_amazon_nova_canvas_image_gen. Nova Canvas is the model the reactivated CI account (888602223428) is entitled to for image generation (see the revert in #29326), it has cost-tracking pricing inmodel_prices_and_context_window.json(output_cost_per_image0.06), and its EOL date is September 30, 2026, so it exercises the same live transform + cost-tracking path the Titan test didThis keeps the test live and hitting the real Bedrock API rather than reverting to a mock
Pre-Submission checklist
@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
This environment has no AWS credentials, so I could not run the live Nova Canvas call locally; CI runs against the reactivated account that is entitled to Nova Canvas image generation and will exercise it. The root cause is confirmed directly from the AWS model lifecycle doc:
Type
✅ Test
Link to Devin session: https://app.devin.ai/sessions/a29c3cb0287344c380f488a20576beba
Requested by: @mateo-berri