[LintDiff] simplify CI/CD deployment / change management

[konrad-jamrozik]

Currently, when working with LintDiff, we leverage rush change (part of rush prep) and rush version --bump (part of deployment to prod). This is confusing and hard to get right without good understanding what rush is doing. Recently I had to yet-again clean up the versioning state of the repo in recent PR Azure/azure-openapi-validator#657. For example, one needs to do following set of changes each time deploying to prod:

rush change 
# <describe the changes interactively>
rush version --bump
# <revert changes to the packages that have not been modified by given PR>
# <verify the git diff looks good>
# At this point, this command would fail: rush change --verify
rush change --bump-type none --message "Update changelog via 'rush version --bump" --bulk
git add *
git commit -am "rush change && rush version --bump"
git push
rush change --verify # this should pass

The goal of this work item is to stop using these commands: instead of that the documentation will explain how to manually update relevant changelog.md and relevant package.json files. As part of this work, the call to rush change --verify from the LintDiff PR CI needs to be removed.

This work item subsumes large part of the previous work item of:

• [LintDiff] CI improvements - check for rush build run and prevent local breakage #6141

Related:

• [LintDiff] Pin version of @microsoft.azure/openapi-validator-rulesets #7613

@weshaggard @mikeharder @AkhilaIlla @rkmanda @bdefoy FYI

[konrad-jamrozik]

I discussed this with @mikeharder.

First, we observed that rush change usage is a dependency on the beta/staging package release mechanism, due to this code:

https://github.com/Azure/azure-openapi-validator/blob/3391350b6651dd51bc4826d101b755c1cf3c6734/eng/scripts/prerelease.mjs#L112

    await addPrereleaseNumber(changeCounts, packages);
    if (Object.keys(changeCounts).length) {
       updateOpenapiValidatorPck()
    }
}

https://github.com/Azure/azure-openapi-validator/blob/3391350b6651dd51bc4826d101b755c1cf3c6734/eng/scripts/prerelease.mjs#L74-L76

    const newVersion = changeCount === 0
        ? packageInfo.version // Use existing version. Bug in rush --partial-prerelease not working
        : `${packageJsonContent.version}.${changeCount}`;

i.e. if there are no files generated with rush change, the beta package release pipeline will not append any suffix to package version. Example log:

Adding prerelease number
Setting version for @microsoft.azure/openapi-validator-core to '1.0.4'
Setting version for @microsoft.azure/openapi-validator to '2.1.7'
Setting version for @microsoft.azure/openapi-validator-rulesets to '1.3.4'
Setting version for openapi-validator-regression to '1.0.0'

And in staging release / publish to npm:

2024-02-01T23:41:01.3321495Z publishing @microsoft.azure/[email protected] from /home/vsts/work/r1/a/_Staging version build for Azure.azure-openapi-validator/drop/packages/rulesets/microsoft.azure-openapi-validator-rulesets-1.3.4.tgz
2024-02-01T23:41:01.3322137Z running: npm publish "/home/vsts/work/r1/a/_Staging version build for Azure.azure-openapi-validator/drop/packages/rulesets/microsoft.azure-openapi-validator-rulesets-1.3.4.tgz" --access=public --tag beta

as a result of this, for example, rulesets 1.3.4 was published, tagged with beta, which made the production release pipeline skip publishing it, as can be seen here

2024-02-02T00:02:51.6938057Z @microsoft.azure/[email protected] from ../_Azure.azure-openapi-validator/drop/packages/rulesets/microsoft.azure-openapi-validator-rulesets-1.3.4.tgz is skipped
2024-02-02T00:02:51.6938979Z @microsoft.azure/[email protected] from ../_Azure.azure-openapi-validator/drop/packages/azure-openapi-validator/autorest/node_modules/@microsoft.azure/openapi-validator-rulesets/microsoft.azure-openapi-validator-rulesets-1.3.4.tgz is skipped

We still want to simplify the process. The step we are planning to take:

• disable the production release
• make the staging/beta pipeline always release the package with beta tag and without any version suffix

This way:

• no rush change files will be required
• if a PR author doesn't want to publish new package version upon PR merge, they need to ensure they didn't update the package.json
• if we want to actually deploy a change to production, we will make the latest tag point to the package that has beta tag pointing to it. This can be accomplished with js - npm-admin-tasks pipeline. See this doc.

Going forward, we plan to completely remove rush from the repository and then update the package versioning to be more inline with standard node best practices. We treat this as an interim solution.

[konrad-jamrozik]

Related doc update work:

• LintDiff: document the new, simpler deployment process azure-openapi-validator#683