Release v1.27.1 - #3646
Conversation
Python API breakage checks — ✅ PASSEDResult: ✅ PASSED |
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
|
Evaluation Triggered
|
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
⚠️ QA Report: PASS WITH ISSUES
SDK behavior was exercised directly: Gemini formatting no longer emits explicit cache_control markers, Anthropic explicit caching still does, and installed package metadata reports 1.27.1.
Does this PR achieve its stated goal?
Yes. The stated goal is a patch release v1.27.1 containing the Gemini prompt-caching fix without the rest of post-1.27.0 main; running the SDK against the v1.27.0 baseline produced two Gemini cache_control markers, while the PR branch at 430ef646 produced zero for the same litellm_proxy/gemini-3.1-pro-preview input. The release version metadata also reports 1.27.1 for all four editable packages. CI was not fully green at the time of QA, so this is a functional pass with release-readiness caveat.
| Phase | Result |
|---|---|
| Environment Setup | ✅ make build completed and installed the editable packages |
| CI Status | gh pr checks showed 1 failing, 14 successful, 2 skipped, 23 pending |
| Functional Verification | ✅ Gemini markers removed; Anthropic markers preserved; package metadata is 1.27.1 |
Functional Verification
Test 1: Gemini explicit cache markers are removed
Step 1 — Reproduce / establish baseline without the fix:
Ran the v1.27.0 SDK source in a temporary worktree with the same user-style SDK operation: instantiate LLM(model="litellm_proxy/gemini-3.1-pro-preview", caching_prompt=True), format a system/user/assistant/user message history, and count emitted cache_control keys.
git worktree add --detach /tmp/oh-sdk-v1270 v1.27.0
OPENHANDS_SUPPRESS_BANNER=1 PYTHONPATH=/tmp/oh-sdk-v1270/openhands-sdk \
/home/runner/work/software-agent-sdk/software-agent-sdk/pr-repo/.venv/bin/python - <<'PY'
# Instantiate Gemini LLM with caching_prompt=True, call format_messages_for_llm,
# count cache_control markers in formatted message/block payloads.
PYObserved output:
baseline_tag=v1.27.0
model=litellm_proxy/gemini-3.1-pro-preview
is_caching_prompt_active=True
cache_control_marker_count=2
cache_control_markers=[(0, 0, {'type': 'ephemeral'}), (3, 0, {'type': 'ephemeral'})]
roles=system,user,assistant,user
This confirms the old v1.27.0 behavior: Gemini was treated as explicit prompt-caching-active and emitted cache markers on both the system prefix and the last user turn.
Step 2 — Apply the PR's changes:
The repository was already checked out on rel-1.27.1 at 430ef6469e7d146876a6f5af1f1f18636a4d696f.
Step 3 — Re-run with the fix in place:
Ran the equivalent SDK operation on the PR branch:
uv run python - <<'PY'
# Same Gemini LLM/message formatting operation as the baseline run.
PYObserved output:
model=litellm_proxy/gemini-3.1-pro-preview
is_caching_prompt_active=False
cache_control_marker_count=0
cache_control_markers=[]
roles=system,user,assistant,user
This shows the PR delivers the core fix: with caching_prompt=True, Gemini no longer activates explicit prompt caching and no cache_control markers are emitted in the payload that the SDK formats for LiteLLM.
Test 2: Anthropic explicit prompt caching still emits markers
Ran the same SDK formatting operation on the PR branch with model="claude-sonnet-4-20250514":
OPENHANDS_SUPPRESS_BANNER=1 uv run python - <<'PY'
# Instantiate Anthropic LLM with caching_prompt=True, call format_messages_for_llm,
# count cache_control markers in formatted message/block payloads.
PYObserved output:
model=claude-sonnet-4-20250514
is_caching_prompt_active=True
cache_control_marker_count=2
cache_control_markers=[(0, 0, {'type': 'ephemeral'}), (3, 0, {'type': 'ephemeral'})]
roles=system,user,assistant,user
This checks the nearest related behavior: explicit cache markers remain active for an Anthropic model, so the Gemini exclusion did not disable the intended Anthropic path.
Test 3: Installed release package metadata is 1.27.1
Ran:
OPENHANDS_SUPPRESS_BANNER=1 uv run python - <<'PY'
from importlib.metadata import version
for package in [
"openhands-sdk",
"openhands-agent-server",
"openhands-tools",
"openhands-workspace",
]:
print(f"{package}={version(package)}")
PYObserved output:
openhands-sdk=1.27.1
openhands-agent-server=1.27.1
openhands-tools=1.27.1
openhands-workspace=1.27.1
This confirms a user installing/building this branch sees the expected patch-release version across the editable packages.
Issues Found
- 🟠 Release readiness caveat: CI was not fully green when checked (
Persisted settings compatibility checks/Persisted settingsfailing; 23 checks pending). No functional issue was found in the manually exercised Gemini caching behavior, but this release PR should wait for CI to finish/resolve before publication.
This QA review was created by an AI agent (OpenHands) on behalf of the user.
Coverage Report •
|
||||||||||||||||||||||||||||||
|
@ak684 It seems we already tagged? We should not tag before the release PR is ready. Also the tag has a failure in CI. 😢 Could we please announce a release PR in the Slack channel? And notify people |
Sounds like this is because the tagged version was first |
enyst
left a comment
enyst
left a comment
There was a problem hiding this comment.
Just a moment Alona, to figure this out. I think you started the release without OpenHands using its skill?
Normally we tell it to start a release PR, and it knows to also label it and stuff
|
@ak684 @VascoSch92 We merged the minor release, and it contains this commit. Do you still a special patch only for that commit? If yes, maybe we can tell OH exactly to make a release PR for patching 1.27.0 with this commit, to make 1.27.2, and have the process play out. The workflows are almost fully automated, and please note: it will also tag when merged. I'm guessing the initial tag that already went out, well, it will be out there. 😅 |
|
oof sorry team! I let my agent run to try and get a release out for OHE with specific fixes and walked away and it clearly caused havoc here, i apologize! |
|
Hah, tiny bit o' havoc 😅 |
4 participants
Patch release: v1.27.0 + cherry-pick of #3586 (fix(llm): don't emit explicit cache_control markers for Gemini) only.
Cut for the OpenHands Enterprise 1.39.0 release so OHE installs get the Gemini prompt-caching fix without the rest of post-1.27.0 main. Release will be published from this branch per the README-RELEASE process; this PR is for visibility and is not intended to merge to main (version files would conflict with the next main release).
Agent Server images for this PR
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdknikolaik/python-nodejs:python3.13-nodejs22-slimgolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:430ef64-pythonRun
All tags pushed for this build
About Multi-Architecture Support
430ef64-python) is a multi-arch manifest supporting both amd64 and arm64430ef64-python-amd64) are also available if needed