[Internal] Pipelines: Fixes fork-runnable CI#5854
Closed
NaluTripician wants to merge 4 commits into
Closed
Conversation
The MultiRegion and MultiMaster EmulatorTests jobs were causing every
fork PR to red-X two required checks because Azure DevOps does not mount
secret variables (COSMOSDB_MULTI_REGION, COSMOSDB_MULTIMASTER) on builds
triggered by fork PRs. With the connection strings arriving empty, the
CosmosClient constructor failed before any test logic ran.
Following the same pattern used by azure-sdk-for-rust, azure-sdk-for-java,
and azure-sdk-for-python, the live-account jobs are now structurally
separated:
* templates/build-test.yml: no longer defines or accepts live-account
parameters. Public/fork PR pipeline (azure-pipelines.yml) consumes
only this template now and is fully fork-safe.
* templates/build-test-live.yml (new): contains just the two
live-account jobs (MultiRegion, MultiMaster), parameterized by the
connection-string secrets.
* azure-pipelines-live-account.yml (new): on-demand top-level pipeline
invoking build-test-live.yml. Configured (via ADO UI) to require a
team-member /azp run comment to dispatch on fork PRs.
* azure-pipelines-official.yml and azure-pipelines-rolling.yml now
invoke both templates so the live-account tests still run on the
release and rolling cadences with their existing secret access.
CONTRIBUTING.md and docs/builds-and-pipelines.md document the new layout,
the /azp run trigger for fork PRs, and the one-time ADO + GitHub
configuration steps needed before this lands on main.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NaluTripician
requested review from
FabianMeiswinkel,
Pilchie,
jeet1995,
kirankumarkolli,
kundadebdatta,
kushagraThapar,
sboshra and
xinlian12
as code owners
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
NaluTripician
changed the title
NaluTripician
changed the title
# Conflicts: # azure-pipelines-rolling.yml
Resolves CONTRIBUTING.md conflict by placing the new fork-CI documentation (### How CI works for pull requests from forks, ### Working on the pipeline split itself) under ## Contribution flow, ahead of the new ## Changelog entry section that was added to main in PR #5864. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #5298 to fix the recurring pattern where fork-submitted PRs always red-X two required CI checks (
EmulatorTests Release - MultiMaster,EmulatorTests Release - MultiRegion) regardless of what the PR actually changes.Root cause
Both jobs depend on the
$(COSMOSDB_MULTI_REGION)and$(COSMOSDB_MULTIMASTER)secret variables. Azure DevOps does not mount secret variables on builds triggered by pull requests from forks (security default). The connection strings arrive empty, andCosmosClientconstruction fails before any test logic runs.Fix
Structurally separate live-account CI from fork-runnable CI, matching the pattern already used by
azure-sdk-for-rust(sdk/cosmos/ci.yml+eng/pipelines/pullrequest.yml+archetype-sdk-client.ymlif and(eq(System.TeamProject, ''internal''), eq(RunLiveTests, ''true''))),azure-sdk-for-java(cosmos-sdk-client.yml/archetype-sdk-client.yml), andazure-sdk-for-python:templates/build-test.ymlno longer defines or accepts live-account params or jobs. The default PR pipeline (azure-pipelines.yml) consumes only this template now and is fully fork-safe.templates/build-test-live.yml(new) — contains just theMultiRegion+MultiMasterjobs, parameterized by the connection-string secrets.azure-pipelines-live-account.yml(new) — on-demand top-level pipeline that invokesbuild-test-live.yml.pr: noneso it never auto-runs on fork PRs; maintainers dispatch it on a PR with/azp run dotnet-v3-live-account.azure-pipelines-official.ymlandazure-pipelines-rolling.ymlinvoke bothbuild-test.ymlandbuild-test-live.ymlso live coverage on the release / rolling cadences is preserved.CONTRIBUTING.mdgains a "How CI works for pull requests from forks" section explaining the/azp runflow contributors will see, plus a "Working on the pipeline split itself" guidance section.docs/builds-and-pipelines.mdgains a "Live account validation" section describing the new pipeline and a "Manual ADO setup" runbook for the one-time configuration that cannot be expressed in YAML.Validation
yaml.safe_load.MemberInitExpression#5298 unit tests) passes locally; that PR''s only test-bearing job (dotnet-v3-ci (Microsoft.Azure.Cosmos.Tests)) was already green — confirmingMultiRegion/MultiMasterfailures were unrelated to the change.Required follow-up ADO + GitHub configuration (cannot be in YAML)
These steps must be performed by a repo administrator after merge (also documented in
docs/builds-and-pipelines.md#manual-ado-setup):dotnet-v3-live-accountincosmos-db-sdk-public, pointing atazure-pipelines-live-account.yml.COSMOSDB_MULTI_REGIONandCOSMOSDB_MULTIMASTERsecret variables to that pipeline definition./azp run dotnet-v3-live-account).dotnet-v3-ciPR-validation settings: Build PRs from forks ON, Make secrets available OFF, Require a comment OFF.dotnet-v3-live-accountto required status checks in GitHub branch protection formain.dotnet-v3-cishould run automatically and green;dotnet-v3-live-accountshould show pending until a maintainer comments/azp run dotnet-v3-live-account.Type of change
Closing issues
Related: #5298 (and other fork PRs that hit the same red-X pattern).