Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 4, 2025

HostActivation tests are hitting the 5-minute hang dump timeout on OSX CI despite making steady progress. Tests complete in ~2 minutes locally but CI environments run 2x slower.

Changes:

  • Increased --hangdump-timeout from 5m to 8m in src/installer/tests/Directory.Build.props

This provides adequate headroom for slower CI machines while still catching actual hangs.

Original prompt

This section details on the original issue you should resolve

<issue_title>Host Activation test failures on OSX</issue_title>
<issue_description>## Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1219628
Build error leg or test failing: HostActivation.Tests.WorkItemExecution
Pull request: #121973

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Hang dump timeout of '00:05:00' expired",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1219628
Error message validated: [Hang dump timeout of '00:05:00' expired]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 12/4/2025 3:07:33 AM UTC

Report

Build Definition Test Pull Request
1219965 dotnet/runtime HostActivation.Tests.WorkItemExecution #122136
1219914 dotnet/runtime HostActivation.Tests.WorkItemExecution #117148
1219763 dotnet/runtime HostActivation.Tests.WorkItemExecution #122125
1219716 dotnet/runtime HostActivation.Tests.WorkItemExecution #122168
1219709 dotnet/runtime HostActivation.Tests.WorkItemExecution #122163
1219628 dotnet/runtime HostActivation.Tests.WorkItemExecution #121973

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
6 6 6

Comments on the Issue (you are @copilot in this section)

@agocke My PR almost certainly broke this, but I'm not sure how yet @agocke Actually, I'm not sure I "broke" this in a real sense -- right now the hang dump timeout is kicking in at the 5m mark. But the log shows a steady stream of tests passing all the way up to the 5m mark.

I wonder if this is just that the tests are taking longer than 5m to finish and the hang dump timeout is measuring from the start of the test run.</comment_new>
<comment_new>@agocke
Yeah my uncontended M2 Mac takes 2 min to run the tests, I could easily believe CI is twice as slow. I think we should bump this to ten minutes.</comment_new>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@github-actions github-actions bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Dec 4, 2025
Copilot AI changed the title [WIP] Fix host activation test failures on OSX Increase HostActivation test hang dump timeout to 8 minutes for OSX CI Dec 4, 2025
Copilot AI requested a review from agocke December 4, 2025 08:42
@jkotas jkotas added area-Host and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Dec 4, 2025
@dotnet-policy-service
Copy link
Contributor

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

@agocke agocke marked this pull request as ready for review December 4, 2025 15:59
Copilot AI review requested due to automatic review settings December 4, 2025 15:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR increases the hang dump timeout for host installer tests from 5 minutes to 8 minutes to accommodate slower CI environments, particularly on OSX where HostActivation tests were being incorrectly flagged as hangs despite making steady progress.

Key Changes:

  • Increased --hangdump-timeout parameter from 5m to 8m in test runner configuration

@agocke
Copy link
Member

agocke commented Dec 4, 2025

This happened to coincide with mac queues coming back online, and has suddenly stopped reproducing. My thinking might just be the heavy overloading caused these tests to start timing out. We may not need to change anything

@agocke
Copy link
Member

agocke commented Dec 5, 2025

Nope still happening occasionally, we should just bump the timeout

@agocke agocke requested a review from elinor-fung December 5, 2025 18:53
@elinor-fung
Copy link
Member

elinor-fung commented Dec 5, 2025

I filed microsoft/testfx#7060

On macOS, the --hangdump-timeout option is being considered as for the entire test run. On other platforms, it is per-test. The --blame-hang-timeout option we were using before the xunit v3 update is per-test.

That is why we started hitting this after the switch to xunit v3 and only on macOS (and only on the one where we run Debug), not other platforms (even in Debug).

@agocke agocke merged commit bce6119 into main Dec 5, 2025
91 checks passed
@agocke agocke deleted the copilot/fix-host-activation-tests-osx branch December 5, 2025 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Host Activation test failures on OSX

4 participants