PR-CI: Split Build and Analyze into independent LOC-based batched matrices by m-nash · Pull Request #59444 · Azure/azure-sdk-for-net

m-nash · 2026-05-26T16:30:26Z

Splits PR-CI Build and Analyze matrix generation into two independent LOC-weighted passes:

Build matrix targets ~1.2M LOC/batch (few big batches)
Analyze matrix targets ~300k LOC/batch (more, smaller batches)

This collapses the Build/Analyze wall-clock spread from 21-31 min down to <=7 min across 20/50/100-pkg PRs, with Analyze wall-clock down 23-50% and total agent-minutes roughly flat (-10% / -6% at 50/100 pkg).

See #59443 for full rationale, measurement methodology, and the comparison vs. simply shrinking batch size uniformly.

Artifact publishing

To keep the two matrix-gens fully independent (no ordering dependency), each pass publishes its own artifact:

Build matrix-gen publishes BuildPackagesArtifact (consumed by Build jobs)
Analyze matrix-gen publishes AnalyzePackageInfoArtifact (consumed by Analyze jobs)

Both matrix-gens still run in parallel; analyze jobs no longer wait for Build to publish.

Files changed

eng/pipelines/templates/jobs/ci.yml - parallel Build and Analyze matrix-gens, each with its own publishing artifact
eng/pipelines/templates/steps/pr-matrix-presteps.yml - new EnableLocWeighting / LocTarget / PublishArtifact params; LOC weighting at end
eng/pipelines/templates/jobs/batched-build.yml (new)
eng/pipelines/templates/jobs/batched-analyze.yml (new) - threads ArtifactName through to analyze.yml
eng/pipelines/templates/steps/analyze.yml - parameterized download artifact name (default preserves prior behavior)
eng/scripts/Get-PackageLocWeights.ps1 (new)
eng/scripts/Apply-WeightedBatching.ps1 (new)

Measurement PRs

Set A (baseline, target main): [DO NOT MERGE] CI measurement: 20pkg - Set B #59401 / [DO NOT MERGE] CI measurement: 50pkg - Set B #59402 / [DO NOT MERGE] CI measurement: 100pkg - Set B #59403
Set B (treatment, target this branch): [DO NOT MERGE] CI measurement: 20pkg - Set A #59404 / [DO NOT MERGE] CI measurement: 50pkg - Set A #59405 / [DO NOT MERGE] CI measurement: 100pkg - Set A #59406

Replaces single batched-build-analyze matrix with two parallel matrix-gens, each tuned independently: - Build: LocTargetSeconds=1,200,000 (larger batches, fewer jobs, lower fixed-overhead) - Analyze: LocTargetSeconds=300,000 (smaller batches, more parallelism) Adds pr-matrix-presteps.yml params LocTargetSeconds (default 474000) and PublishArtifact (default true) so each matrix-gen can configure LOC target independently and only the first publishes BuildPackagesArtifact (avoiding duplicate-artifact collision). Adds Get-PackageLocWeights.ps1 (LOC counter) and Apply-WeightedBatching.ps1 (LPT bin-packing) which together drive the LOC-weighted bin-packing. Removes the now-unused batched-build-analyze.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates the PR-CI pipeline to generate separate Build and Analyze job matrices using LOC-weighted batching, so Build runs in fewer/larger batches and Analyze runs in more/smaller batches—aiming to reduce wall-clock skew between the two stages.

Changes:

Split PR matrix generation into two independent passes: Build (~1.2M LOC target) and Analyze (~300k LOC target).
Added PowerShell scripts to compute per-package LOC and apply weighted LPT bin-packing to consolidate PackageInfo files into balanced batches.
Introduced dedicated job templates for batched Build vs batched Analyze.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
eng/pipelines/templates/jobs/ci.yml	Runs two independent PR matrix generations (Build + Analyze) and wires them to separate batched job templates.
eng/pipelines/templates/steps/pr-matrix-presteps.yml	Adds optional LOC-weighting + optional artifact publishing during PR matrix pre-generation.
eng/pipelines/templates/jobs/batched-build.yml	Build-only batched job template (Analyze removed).
eng/pipelines/templates/jobs/batched-analyze.yml	New Analyze-only batched job template (Linux pool).
eng/scripts/Get-PackageLocWeights.ps1	New script to compute per-package C# LOC weights from `src/*/.cs`.
eng/scripts/Apply-WeightedBatching.ps1	New script to apply weighted LPT batching and consolidate PackageInfo files per batch.

Eliminates the artifact-download race between the parallel build and analyze matrix-gen stages. Build matrix-gen continues to publish BuildPackagesArtifact for build jobs; analyze matrix-gen now publishes its own AnalyzePackageInfoArtifact for analyze jobs. Both matrix-gens remain fully parallel - no wall-clock cost. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The parameter values are LOC counts, not seconds. The 'Seconds' suffix was a misleading carryover from when the script was time-based. Renames the YAML param, the script param, internal variables, docstrings, and the display/log messages so future readers don't misconfigure them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

…rices (Azure#59444) * Split Build/Analyze into independent LOC-weighted matrices Replaces single batched-build-analyze matrix with two parallel matrix-gens, each tuned independently: - Build: LocTargetSeconds=1,200,000 (larger batches, fewer jobs, lower fixed-overhead) - Analyze: LocTargetSeconds=300,000 (smaller batches, more parallelism) Adds pr-matrix-presteps.yml params LocTargetSeconds (default 474000) and PublishArtifact (default true) so each matrix-gen can configure LOC target independently and only the first publishes BuildPackagesArtifact (avoiding duplicate-artifact collision). Adds Get-PackageLocWeights.ps1 (LOC counter) and Apply-WeightedBatching.ps1 (LPT bin-packing) which together drive the LOC-weighted bin-packing. Removes the now-unused batched-build-analyze.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use separate AnalyzePackageInfoArtifact for analyze stage Eliminates the artifact-download race between the parallel build and analyze matrix-gen stages. Build matrix-gen continues to publish BuildPackagesArtifact for build jobs; analyze matrix-gen now publishes its own AnalyzePackageInfoArtifact for analyze jobs. Both matrix-gens remain fully parallel - no wall-clock cost. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename LocTargetSeconds -> LocTarget (and -TargetSeconds -> -Target) The parameter values are LOC counts, not seconds. The 'Seconds' suffix was a misleading carryover from when the script was time-based. Renames the YAML param, the script param, internal variables, docstrings, and the display/log messages so future readers don't misconfigure them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove time-unit suffixes from LOC weight log lines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Resolve DirectoryPath against RepoRoot in Get-PackageLocWeights Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify LOC weights key is ArtifactName Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify Target is a goal not a hard cap Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename EnableBuildWeighting -> EnableLocWeighting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Make Apply-WeightedBatching synopsis job/unit agnostic Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Align WeightsFile help text with ArtifactName key Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 26, 2026 16:30

m-nash requested review from Ben Broderick Phillips (benbp), Patrick Hallisey (hallipr) and Wes Haggard (weshaggard) as code owners May 26, 2026 16:30

Copilot started reviewing on behalf of m-nash May 26, 2026 16:30 View session

JoshLove-msft approved these changes May 26, 2026

View reviewed changes

Copilot AI reviewed May 26, 2026

View reviewed changes

m-nash and others added 2 commits May 26, 2026 10:16

Jesse Squire (jsquire) approved these changes May 26, 2026

View reviewed changes

Comment thread eng/scripts/Get-PackageLocWeights.ps1

m-nash and others added 2 commits May 26, 2026 12:44

Remove time-unit suffixes from LOC weight log lines

3f4aaf6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Resolve DirectoryPath against RepoRoot in Get-PackageLocWeights

fbc0eb3

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

m-nash requested a review from Copilot May 26, 2026 19:46

Copilot started reviewing on behalf of m-nash May 26, 2026 19:46 View session

m-nash requested a review from Jesse Squire (jsquire) May 26, 2026 19:46

Copilot AI reviewed May 26, 2026

View reviewed changes

Comment thread eng/scripts/Get-PackageLocWeights.ps1 Outdated

Comment thread eng/scripts/Apply-WeightedBatching.ps1 Outdated

Comment thread eng/pipelines/templates/steps/pr-matrix-presteps.yml Outdated

Comment thread eng/pipelines/templates/jobs/ci.yml

m-nash and others added 3 commits May 26, 2026 12:59

Clarify LOC weights key is ArtifactName

a679670

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Clarify Target is a goal not a hard cap

4e4eadb

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename EnableBuildWeighting -> EnableLocWeighting

f328047

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Ben Broderick Phillips (benbp) approved these changes May 26, 2026

View reviewed changes

m-nash requested a review from Copilot May 26, 2026 20:10

Copilot started reviewing on behalf of m-nash May 26, 2026 20:10 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

Comment thread eng/scripts/Apply-WeightedBatching.ps1

Comment thread eng/scripts/Apply-WeightedBatching.ps1 Outdated

m-nash and others added 2 commits May 26, 2026 13:26

Make Apply-WeightedBatching synopsis job/unit agnostic

ca50ab0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Align WeightsFile help text with ArtifactName key

890e48b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

m-nash requested a review from Copilot May 26, 2026 20:33

Copilot started reviewing on behalf of m-nash May 26, 2026 20:34 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

Comment thread eng/scripts/Apply-WeightedBatching.ps1

m-nash requested a review from Copilot May 26, 2026 20:48

Copilot started reviewing on behalf of m-nash May 26, 2026 20:48 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

Jesse Squire (jsquire) approved these changes May 26, 2026

View reviewed changes

m-nash merged commit 4564b9f into main May 26, 2026
29 checks passed

m-nash deleted the feature/split-build-analyze-loc branch May 26, 2026 21:17

Wei Hu (live1206) mentioned this pull request Jul 28, 2026

Speed up PR matrix LOC counting #61442

Closed

Navigation Menu

Uh oh!

PR-CI: Split Build and Analyze into independent LOC-based batched matrices - #59444

PR-CI: Split Build and Analyze into independent LOC-based batched matrices#59444
m-nash merged 10 commits into
mainfrom
feature/split-build-analyze-loc

m-nash commented May 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

m-nash commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact publishing

Files changed

Measurement PRs

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

m-nash commented May 26, 2026 •

edited

Loading