PLTF-2295: Upgrade litellm for opus 4.7

[aivong-openhands]

Description

Upgrades LiteLLM from v1.82.3-stable.patch.3 to v1.83.14-stable to match the version we use in SaaS (cloud). Adds support for Claude Opus 4.7 by adding the model to the LiteLLM proxy model list as a workaround given the LiteLLM bug when using the Anthropic provider directly. Also includes a temporary enterprise-server image override (sha-d316113) that fixes org settings 500 errors on upgrade installs (OpenHands/OpenHands#14326).

Changes

• LiteLLM image: v1.82.3-stable.patch.3 → v1.83.14-stable
• Proxy model list: adds claude-opus-4-7 → anthropic/claude-opus-4-7
• v2 migration resolver: enables --use_v2_migration_resolver on LiteLLM startup to prevent schema thrashing during upgrades
• Enterprise-server image override: pins to sha-d316113 to include the org settings PATCH write-path fix (PLTF-2295: org settings PATCH for legacy agent_kind rows OpenHands#14326); to be replaced with a release tag once that PR merges

Why the proxy model entry is required

LiteLLM 1.83.14 has a bug where thinking: {type: enabled} is not correctly translated to the adaptive thinking format when calling Anthropic directly with claude-opus-4-7, causing a BadRequestError from the Anthropic API. The workaround is to route requests through the LiteLLM proxy using the openhands provider (litellm_proxy/claude-opus-4-7), which uses the reasoning_effort parameter path that works correctly in this version.

Without the proxy model entry, the proxy rejects requests with "Invalid model name passed in model=claude-opus-4-7". The upstream fix is tracked in LiteLLM PR #27074.

Why the v2 migration resolver is required

Without --use_v2_migration_resolver, LiteLLM's _resolve_all_migrations logic can thrash the database schema during rolling deployments when two versions contend for the same database. This flag limits migration behavior to prisma migrate deploy only, preventing the diff-and-force recovery path that caused a production incident (PLTF-2292) on April 29, 2026. Customers upgrading from v1.82.3 to v1.83.14 are exposed to this without the flag.

Why --config /etc/litellm/config.yaml is explicitly added

This is not a new LiteLLM 1.83.14 requirement. The upstream litellm-helm chart already starts LiteLLM with --config /etc/litellm/config.yaml by default so it loads the mounted proxy config (including proxy_config.model_list).

In this PR we also need to add --use_v2_migration_resolver. Because the chart's args value replaces the entire default argument list rather than appending to it, we must restate --config /etc/litellm/config.yaml alongside the new flag. Without it, LiteLLM would start without loading the mounted config file and would ignore the configured proxy models.

Why the enterprise-server image override is required

Customers upgrading from an older version have agent_kind='llm' stored in their org's agent_settings DB column (written by the old enterprise server before LLMAgentSettings was renamed to OpenHandsAgentSettings). The PATCH /api/organizations/{id}/settings write path validates the merged settings against OpenHandsAgentSettings without normalizing the legacy value, causing a 500 error that prevents saving org settings — including switching to litellm_proxy/claude-opus-4-7. The fix is in OpenHands/OpenHands#14326. The sha-d316113 override will be replaced with the official release tag once that PR merges.

Usage

In OpenHands org defaults, configure:

• Model: openhands/claude-opus-4-7
• Base URL: http://openhands-litellm:4000

Helm Chart Checklist

• I have updated the version field in Chart.yaml for each modified chart
• I have tested the chart upgrade path from the previous version
• I have verified backwards compatibility with existing values.yaml configurations
• I have updated the chart's README.md if there are any breaking changes or new required values

Changes are only in the Replicated configs not the main charts so no version bump is needed.

Additional Notes

The LiteLLM thinking translation bug affects the anthropic provider path only. The openhands provider path (via the LiteLLM proxy) is unaffected and works correctly with this configuration.

Verified in a Replicated install that this configuration upgrades LiteLLM to 1.83.14. Also allows us to use the Claude Opus 4.7 model via the OpenHands provider to use the proxied model:

Looking at the exported conversation metadata confirms the model used in the test conversation is the proxied model:

Tested the upgrade path from 0.7.3 → 0.7.8 on a Replicated embedded cluster VM. LiteLLM startup logs confirmed the v2 migration resolver was active (Using v2 migration resolver (--use_v2_migration_resolver)), 11 pending migrations were applied cleanly via prisma migrate deploy, and no schema thrashing occurred. The diff-and-force recovery path was not triggered.

Verified the org settings PATCH fix (OpenHands/OpenHands#14326) on both install paths using enterprise-server sha-d316113:

• Fresh install (0.7.8): org created with agent_kind='openhands', org settings saved successfully with litellm_proxy/claude-opus-4-7, conversation completed and confirmed via conversation_metadata.llm_model = litellm_proxy/claude-opus-4-7 in the DB ✅
• Upgrade install (0.7.3 → 0.7.8): org had legacy agent_kind='llm' in DB before upgrade, org settings saved successfully after upgrade with agent_kind normalized to 'openhands' and model updated to litellm_proxy/claude-opus-4-7 in the DB ✅

[all-hands-bot]

🟡 Acceptable with critical concerns that must be addressed before deployment.

This PR upgrades LiteLLM to add Claude Opus 4.7 support, but deploys a version with a known bug requiring a proxy workaround. The Helm Chart Checklist is incomplete and there's no evidence the changes were tested.

[all-hands-bot]

Code Review Analysis

[CRITICAL ISSUES]

• [PR Description] Missing Evidence: The PR description lacks an Evidence section proving the changes work. According to the active review instructions, you must include:

  • For backend/config changes: The exact commands used to test the configuration end-to-end and the resulting output
  • Screenshots/videos showing Claude Opus 4.7 working through the proxy
  • Tests alone don't count - we need to see real runtime verification
  • Without evidence, we can't verify that the workaround actually resolves the LiteLLM bug

• [PR Description] Helm Chart Checklist Incomplete: All items in the Helm Chart Checklist are unchecked:

  • Chart version field not updated
  • Upgrade path not tested
  • Backwards compatibility not verified
  • README.md not updated for any breaking changes

  This is a deployment configuration change and must complete the checklist before merging.

• [replicated/openhands.yaml, Line 139] Known Bug Deployment: Per the PR description, v1.83.14 has a known bug where thinking: {type: enabled} translation fails. Deploying a version with known bugs is a 🔴 HIGH risk operation. Before proceeding:

  1. Document which teams/services are impacted if the workaround fails
  2. Provide a rollback plan (can we easily revert to v1.82.3?)
  3. State the ETA for the upstream fix (PR #27074)
  4. Verify all other LiteLLM functionality remains stable across the version gap

[IMPROVEMENT OPPORTUNITIES]

• [replicated/openhands.yaml, Lines 178-181] Documentation Gap: The model entry at line 178 is a workaround for the bug, but this isn't documented in the YAML itself. Future maintainers won't know why this configuration exists. Add an inline comment:

# Workaround for LiteLLM 1.83.14 bug: routes through proxy to use reasoning_effort path
# instead of broken thinking parameter translation. Tracked in: https://github.com/BerriAI/litellm/pull/27074
# TODO: Re-evaluate this configuration once the upstream fix is released
- model_name: "claude-opus-4-7"
  litellm_params:
    model: "anthropic/claude-opus-4-7"
    api_key: os.environ/ANTHROPIC_API_KEY

• [PR Description] Version Gap Analysis Missing: The upgrade spans from 1.82.3 to 1.83.14. You should review:
  • LiteLLM release notes for v1.82.4 through v1.83.14
  • Any breaking changes or deprecated features
  • Security fixes or CVEs addressed
  • Behavioral changes that could affect existing functionality

[TESTING GAPS]

• [PR Description] No Evidence Section: Add an Evidence section to the PR description with:

  • Command output showing Claude Opus 4.7 successfully processing a request through the proxy
  • Configuration verification (e.g., curl to LiteLLM proxy showing the model is registered)
  • Test showing the workaround bypasses the known bug
  • Example:

    # Test that proxy recognizes the model
    curl -X GET http://openhands-litellm:4000/models
    
    # Test actual inference through proxy
    curl -X POST http://openhands-litellm:4000/chat/completions \
      -H "Content-Type: application/json" \
      -d '{"model": "claude-opus-4-7", "messages": [...], "thinking": {"type": "enabled"}}'

• [Repository] No Integration Tests: While this repository may not have testing infrastructure for LiteLLM proxy configuration, consider:

  • Can we add a smoke test that validates the proxy starts and recognizes the configured models?
  • Should we have a test environment where this configuration is validated before production deployment?

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🔴 HIGH

This PR presents HIGH risk due to:

1. Critical dependency upgrade without testing evidence (LiteLLM is core infrastructure)
2. Known bug in new version requiring workaround (bug severity unclear, no timeline for fix)
3. Helm Chart compliance gaps (checklist incomplete, no version bump mentioned)
4. Version gap (1.82.3 → 1.83.14) with unreviewed release notes and potential breaking changes
5. Production impact scope (affects all LiteLLM proxy users, unclear blast radius if workaround fails)

Recommendation: Do not auto-merge. Request review from:

• Infrastructure/Platform team to validate the deployment configuration
• A human architect to assess whether waiting for the upstream bug fix (PR #27074) is safer than deploying this workaround
• Security team to review release notes for CVEs between 1.82.3 and 1.83.14

Alternative safer approach:

• Wait for LiteLLM upstream fix and upgrade to the patched version
• Or deploy to a staging environment first with comprehensive monitoring before production

VERDICT:

❌ Needs rework: While the code changes are minimal and structurally sound, critical process requirements are missing (Evidence, Helm Chart Checklist), and deploying a known-buggy version with a workaround presents unacceptable production risk without proper validation and documentation.

KEY INSIGHT:

The technical implementation is straightforward, but the risk profile is elevated by deploying known-buggy infrastructure code (LiteLLM) without testing evidence or a timeline for when the proper fix will be available—a workaround is only acceptable if its failure modes are well-understood and documented.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

[all-hands-bot]

🟡 Acceptable - Pragmatic solution with documented workaround.

All previous review concerns have been addressed:

• Inline comment added explaining the temporary workaround (commit 80bc0e2)
• Evidence provided via screenshots showing successful deployment and Claude Opus 4.7 usage
• Helm Chart Checklist explained (not applicable for Replicated configs)
• Version alignment with SaaS environment documented

The implementation is straightforward:

• Version bump to v1.83.14-stable to match SaaS
• Model proxy entry with clear temporary workaround comment
• Upstream fix tracked in LiteLLM PR #27074

[RISK ASSESSMENT]

⚠️ Risk Assessment: 🟡 MEDIUM

This change upgrades LiteLLM across minor versions (v1.82.3 → v1.83.14) and includes a workaround for a known bug where direct Anthropic API calls fail for Claude Opus 4.7. Risk is mitigated by:

1. Version already deployed and validated in the SaaS environment
2. Workaround clearly documented with inline comments and tracked upstream fix (LiteLLM PR #27074)
3. Evidence shows successful deployment and usage in Replicated install
4. Changes isolated to Replicated configs (not core Helm charts)
5. Bug only affects direct Anthropic provider path; proxy path (used by this workaround) works correctly

Risk factors elevating to MEDIUM:

• Infrastructure-level change affecting LLM proxy configuration (moderate blast radius)
• Dependency upgrade with known bug requiring workaround
• Core system impact on LLM access layer

VERDICT:
✅ Worth merging - Pragmatic solution to enable Claude Opus 4.7 support with well-documented workaround

KEY INSIGHT:
This PR prioritizes delivering Claude Opus 4.7 support now with a documented workaround rather than waiting for upstream fix - a reasonable pragmatic trade-off given SaaS validation and tracked remediation path.

[all-hands-bot]

🟡 Acceptable - Pragmatic workaround with strong testing evidence, but one startup argument needs clarification.

The upgrade addresses real production issues (Claude Opus 4.7 support + schema thrashing fix) and follows pragmatic engineering with a well-documented workaround. Testing evidence is strong. One configuration detail needs explanation before merge.

[aivong-openhands]

Going to set this back to draft as an incremental upgrade revealed org settings 500s when trying to set the default model to the proxied opus model

[all-hands-bot]

🟡 Acceptable - Pragmatic upgrade with comprehensive testing evidence.

All previous review concerns have been addressed. The LiteLLM version upgrade, migration resolver flag, and proxy workaround are well-documented and tested. Configuration changes are straightforward.

[RISK ASSESSMENT]
⚠️ Risk Assessment: 🟡 MEDIUM

Infrastructure change affecting critical LLM inference path with multiple temporary workarounds (SHA override + proxy model). Risk is mitigated by: comprehensive testing evidence covering both fresh and upgrade install paths, SaaS version alignment (already production-proven at v1.83.14-stable), and clear rollback strategy to v1.82.3-stable.patch.3. Temporary workarounds are documented with upstream fix tracking.

VERDICT:
✅ Worth merging: Pragmatic solution to production needs

KEY INSIGHT:
Bundling related LiteLLM infrastructure changes with production validation reduces deployment complexity despite multiple moving parts.

[all-hands-bot]

🟢 Good taste - Pragmatic infrastructure upgrade with comprehensive testing.

All previous review concerns have been addressed. The LiteLLM version upgrade, migration resolver flag, and proxy workaround are well-documented with clear tracking for permanent fixes:

• Proxy model entry: temporary workaround for LiteLLM thinking translation bug (tracked in LiteLLM PR #27074)
• Enterprise-server SHA override: fixes org settings 500 errors on upgrade (tracked in OpenHands/OpenHands#14326)
• Migration resolver: prevents schema thrashing during rolling deployments (addresses PLTF-2292)

Testing evidence is excellent: screenshots of working config, upgrade path validation (0.7.3→0.7.8), and verification of both fresh install and upgrade scenarios.

[RISK ASSESSMENT]
⚠️ Risk Assessment: 🟡 MEDIUM

Infrastructure change affecting critical LLM inference path with multiple temporary workarounds. Risk is mitigated by:

• Comprehensive end-to-end testing on both install paths
• Clear documentation and tracking of temporary measures
• Version alignment with production SaaS environment (v1.83.14)
• Successful migration validation (11 migrations applied cleanly)

VERDICT:
✅ Worth merging - Solves real production problems with well-tested pragmatic solutions.

KEY INSIGHT:
Temporary workarounds for upstream bugs are acceptable when properly documented, tested, and tracked - this PR demonstrates that pattern well.