submit all helix items in single job - #84873
Merged
dibarbet merged 3 commits intoAug 18, 2026
Merged
Conversation
|
Azure Pipelines: Successfully started running 2 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
dibarbet
force-pushed
the
dibarbet-combine-helix-submissions
branch
from
August 13, 2026 01:13
1ec4e20 to
63476c0
Compare
|
Azure Pipelines: Successfully started running 2 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
dibarbet
changed the title
dibarbet
commented
Aug 13, 2026
| } | ||
| } | ||
|
|
||
| private static string GetTestRunName(Options options) |
Member
Author
There was a problem hiding this comment.
instead of getting the test configuration name based on the job name that queued it (since we no longer have a job for each test configuration), we derive a stable job name based on the input parameters.
this becomes the test run name in AzDo (with the queue appended):
Member
At a glance that looks like the right amount of tests |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR consolidates Windows Helix test submissions so a single job (per build kind) submits multiple Helix configurations sequentially, reducing Azure DevOps agent usage. To preserve accurate historical scheduling data despite changed job/stage names, it derives a stable test run name from RunTests submission parameters and updates the AzDO history lookup to query by that run name.
Changes:
- Update RunTests test history lookup to identify the relevant AzDO test run by a stable Helix test run name (instead of phase/stage name).
- Derive and flow a stable
TestRunNamePrefixinto Helix submissions so historical test duration data can be scoped to a specific configuration. - Refactor the Windows test stage to submit multiple Helix runs from a single job, preserving per-submission logs/payloads.
Show a summary per file
| File | Description |
|---|---|
| src/Tools/RunTests/TestHistoryManager.cs | Switch history lookup from phase-based filtering to stable test-run-name-based filtering. |
| src/Tools/RunTests/Options.cs | Remove PhaseName option wiring now that history lookup no longer depends on AzDO phase name. |
| src/Tools/RunTests/HelixTestRunner.cs | Generate stable test run name and use it as the Helix TestRunNamePrefix; keep known env-var tokens consistent. |
| src/Tools/RunTests/AzdoClient.cs | Rename/run lookup API to fetch a test run by exact name (case-insensitive). |
| eng/pipelines/test-windows-job.yml | Submit multiple Helix configurations in one job; preserve logs/payloads between sequential submissions. |
| azure-pipelines.yml | Replace multiple Windows Helix test jobs with a single job that iterates through testRuns. |
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 1
- Review effort level: Lite
Comment on lines
+84
to
+85
| Get-ChildItem -Path $logDirectory -Filter 'workitem_*' -Directory | | ||
| Move-Item -Destination $submissionLogDirectory -Force |
JoeRobich
approved these changes
Aug 13, 2026
akhera99
approved these changes
Aug 14, 2026
This was referenced Aug 18, 2026
JoeRobich
added a commit
that referenced
this pull request
Sep 4, 2026
## Summary - backport the Roslyn `RunTests` changes needed for the standalone Helix Job Monitor - consolidate Windows Helix submissions and add monitor jobs for Windows, Linux, and macOS - pin the monitor tool to the Arcade `release/10.0` build used by #85150 This PR is stacked on #85150 so the diff contains only the test runner and CI infrastructure changes. After #85150 merges, this PR can be retargeted to `release/dev18.0`. Backports the relevant behavior from #84810, #84872, #84873, and #85070. ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/dotnet/roslyn/pull/85153) --------- Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f9174b1-67dc-4def-b0c1-3b8e1746e681
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Now that the helix test jobs only submit to helix and do not wait for completion, we can instead run a single job (per build kind) that submits all the test configurations to helix. This reduces the number of AzDo machines we have to acquire.
However - it does mean that the stage / phase names change which breaks test history lookup (would conflate tests with the same name that run in multiple test configurations). To fix that we derive a stable name for the test configuration based on the RunTests parameters that we use when submitting the run to helix. Then we can query AzDo for tests matching that run name when looking for test history to scope them to only tests for that test configuration.
Microsoft Reviewers: Open in CodeFlow