Support pool-specific PR job batch sizes - #16674
Conversation
|
Azure Pipelines: Successfully started running 1 pipeline(s). 63 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
The following pipelines have been queued for testing: |
|
Azure Pipelines: Successfully started running 1 pipeline(s). 63 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds a per-agent-pool override mechanism for PR job batching in the shared generate-job-matrix.yml template, enabling consuming repos to tune PackageInfo grouping based on pool performance characteristics while preserving the existing global default.
Changes:
- Introduces a new
PRJobBatchSizeByPoolparameter (object map) with{}default. - Computes an effective
$batchSizeper pool at runtime, falling back toPRJobBatchSizewhen no override (or an unset runtime macro) is present. - Uses the computed
$batchSizewhen invokingCreate-PrJobMatrix.ps1.
Show a summary per file
| File | Description |
|---|---|
| eng/common/pipelines/templates/jobs/generate-job-matrix.yml | Adds per-pool PR job batch size overrides and runtime selection logic before generating PR matrices. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
|
The following pipelines have been queued for testing: |
|
The following pipelines have been queued for testing: |
0da15e7 to
a727f50
Compare
|
The following pipelines have been queued for testing: |
Amp-Thread-ID: https://ampcode.com/threads/T-019fc928-8629-77e7-9030-1412de6b8f36 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
a727f50 to
046548e
Compare
|
The following pipelines have been queued for testing: |
Sync eng/common directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16674 See [eng/common workflow](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/README.md#workflow) --------- Co-authored-by: Wei Lim <weilim@microsoft.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Wei Lim <weikanglim@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sync eng/common directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16674 See [eng/common workflow](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/README.md#workflow) --------- Co-authored-by: Wei Lim <weilim@microsoft.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: Wei Lim <weikanglim@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Add an optional
PRJobBatchSizeByPoolmap togenerate-job-matrix.ymlto configure batch sizes per-pool.We allow override values of
PRJobBatchSizeByPoolto come from runtime variables populated byPreGenerationSteps(which is useful). In cases where no override exists or a runtime macro remains unset, we use the defaultPRJobBatchSizevalue.Motivation
Some consumers preprocess
PackageInfoentries into scheduling buckets whose optimal grouping differs by agent pool. For example, Azure/azure-sdk-for-net#57798 combines three weighted buckets per Linux and Windows job while using one per slower macOS job. This keeps that policy in the consuming repository while exposing only a generic per-pool batch-size capability from the centrally owned template.Existing callers are unchanged because the new map defaults to empty and the current
PRJobBatchSizeremains the fallback.