Skip to content

Fix: Add 180-minute timeout to integration tests to prevent indefinite hangs - #2131

Merged
juanmichelini merged 3 commits into
mainfrom
openhands/fix-integration-test-timeout-2124
Feb 19, 2026
Merged

juanmichelini merged 3 commits into
mainfrom
openhands/fix-integration-test-timeout-2124

Conversation

@juanmichelini

@juanmichelini juanmichelini commented Feb 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR fixes the Integration Test Action that has been failing continuously by adding an explicit 180-minute timeout to the run-integration-tests job.

Problem:
The run-integration-tests job in the integration-runner.yml workflow has been hanging indefinitely (6-8+ hours), causing workflow runs to never complete. While the setup-matrix job was fixed in PR #2125 (litellm import issue), the actual test execution jobs continue to hang, preventing any workflow runs from finishing successfully.

Evidence:

  • Multiple workflow runs have been stuck "in_progress" for 6-8+ hours
  • The only "successful" runs are ones where tests were skipped entirely
  • Historical successful runs from Feb 17 completed in 1-2 minutes per test
  • All runs since Feb 18 have been hanging

Solution:
Added timeout-minutes: 60 to the run-integration-tests job. This ensures:

  • The job will complete within 60 minutes even if tests hang
  • The workflow will finish and report results instead of hanging forever
  • Failed/timed-out tests can be identified and debugged
  • The consolidate-results job can run and post feedback

Testing:
The timeout setting will be validated by triggering a test run with a small number of tests to verify the workflow completes successfully.

Fixes #2124

Checklist

  • If the PR is changing/adding functionality, are there tests to reflect this? - This is a workflow configuration change, no code tests needed
  • If there is an example, have you run the example to make sure that it works? - N/A (workflow change only)
  • If there are instructions on how to run the code, have you followed the instructions and made sure that it works? - Will be verified by triggering the workflow
  • If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name? - N/A (this is a bug fix, not a new feature)
  • Is the github CI passing? - Will be verified after PR creation

@juanmichelini can click here to continue refining the PR


Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:5e111c8-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-5e111c8-python \
  ghcr.io/openhands/agent-server:5e111c8-python

All tags pushed for this build

ghcr.io/openhands/agent-server:5e111c8-golang-amd64
ghcr.io/openhands/agent-server:5e111c8-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:5e111c8-golang-arm64
ghcr.io/openhands/agent-server:5e111c8-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:5e111c8-java-amd64
ghcr.io/openhands/agent-server:5e111c8-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:5e111c8-java-arm64
ghcr.io/openhands/agent-server:5e111c8-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:5e111c8-python-amd64
ghcr.io/openhands/agent-server:5e111c8-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:5e111c8-python-arm64
ghcr.io/openhands/agent-server:5e111c8-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:5e111c8-golang
ghcr.io/openhands/agent-server:5e111c8-java
ghcr.io/openhands/agent-server:5e111c8-python

About Multi-Architecture Support

  • Each variant tag (e.g., 5e111c8-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 5e111c8-python-amd64) are also available if needed

… hangs

The run-integration-tests job has been hanging indefinitely (6-8+ hours) causing
workflow runs to never complete. Adding an explicit timeout of 60 minutes ensures
the job will complete even if tests are stuck or taking too long.

This allows the workflow to finish and report results instead of hanging forever.

Fixes #2124

Co-authored-by: openhands <openhands@all-hands.dev>

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good taste - Simple, pragmatic fix that unblocks CI. LGTM!

) && needs.setup-matrix.result == 'success'
needs: [setup-matrix, post-label-comment, post-dispatch-comment]
runs-on: ubuntu-22.04
timeout-minutes: 60

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Since tests normally complete in 1-2 minutes, consider a shorter timeout like 10-15 minutes to fail faster and get feedback sooner. 60 minutes works but is quite generous.

That said, this is a reasonable upper bound and the change is good to merge as-is. 👍

@juanmichelini juanmichelini changed the title Fix: Add 60-minute timeout to integration tests to prevent indefinite hangs Fix: Add 180-minute timeout to integration tests to prevent indefinite hangs Feb 19, 2026
@juanmichelini
juanmichelini enabled auto-merge (squash) February 19, 2026 21:39
@juanmichelini

juanmichelini commented Feb 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

180 minutes should be plenty

@juanmichelini
juanmichelini merged commit fadc008 into main Feb 19, 2026
19 of 21 checks passed
@juanmichelini
juanmichelini deleted the openhands/fix-integration-test-timeout-2124 branch February 19, 2026 21:42
StressTestor pushed a commit to StressTestor/software-agent-sdk that referenced this pull request Jun 1, 2026
…e hangs (OpenHands#2131)

Co-authored-by: openhands <openhands@all-hands.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration Test Action has been falling continuously all day

3 participants