Skip to content

Conversation

@alzimmermsft
Copy link
Member

Description

Fixes #26140

This PR adds a new pipeline for /eng/code-quality-reports, referred to as linting pipeline, that is ran any time a change is made to linting tooling and configuration. The new pipeline will build all libraries that use azure-core, directly or in it's dependency graph, and run Checkstyle, RevApi, and Spotbugs against the built libraries. This resolves a gap in coverage where previously changes to eng/code-quality-reports would only run the Core pipeline and this pipeline only runs linting validation for the Core libraries. So, unfortunately there is/was a gap in validating linting changes which had caused unexpected linting errors to occur in other SDK groups as they weren't validated. Additionally, now changes made to eng/code-quality-reports will no longer trigger the Core pipelines (which are generally expensive in terms of validation time).

A few additional changes were made to the parent POM to enable this change. Now there are more configuration points in the POM which allow enabling/disabling Checkstyle, RevApi, and Spotbugs to fail on errors reported. This was done as the linting pipeline should validate all libraries completely ignoring any failures and use a post-processing check to determine if the pipeline should be failed (based on any linting error reports existing). There is also a linting error report processor being added as both Checkstyle and Spotbugs use XML which are human readable but it was determined a CSV would be easier for consumption.

In the future, changes made in this PR have made it easier to allow for parallel processing of linting reports. Currently, these are done synchronously as the CLI output is being used in most places to troubleshoot and resolve linting errors found during CI.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@alzimmermsft alzimmermsft added EngSys This issue is impacting the engineering system. Client This issue points to a problem in the data-plane of the library. labels Dec 28, 2021
@alzimmermsft alzimmermsft self-assigned this Dec 28, 2021
@ghost ghost added the Azure.Core azure-core label Dec 28, 2021
@alzimmermsft
Copy link
Member Author

/azp run prepare-pipelines

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@alzimmermsft alzimmermsft merged commit 2231f27 into Azure:main Jan 3, 2022
@alzimmermsft alzimmermsft deleted the AzEng_AddCodeQualityReportsCIValidation branch January 3, 2022 18:09
exclude:
- eng/versioning
- eng/jacoco-test-coverage
- eng/pipelines/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there might be other paths like eng/common we want to add to this list as well.
cc @benbp @hallipr

azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this pull request Nov 9, 2023
Release app microsoft.app 2023 08 01 preview (Azure#26352)

* Adds base for updating Microsoft.App from version preview/2023-05-02-preview to version 2023-08-01-preview

* Updates readme

* Updates API version in new specs and examples

* Add type to Certificate (Azure#25380)

* Add type to Certificate

* lint

* Add gpus property to AvailableWorkloadProfiles. (Azure#25328)

* Add gpus property to AvailableWorkloadProfiles.

* Fix.

* Pauld/re add build and builder (Azure#25442)

* cherry picked the build and builder updates

* some more cleanup

* added arm id format, removed extra properties, and named the resource provisioning states

* Add OT and AI config (Azure#25350)

* Add OT and AI config

* resolve comments

* Add service bind properties to dapr component (Azure#25781)

* dapr component sb-bind definition add

* fix

* ran npx prettier

* npx prettier

---------

Co-authored-by: [email protected] <[email protected]>

* Adding patching scenario ARM API (Azure#25446)

* cherry picked the build and builder updates

* some more cleanup

* Adding patching scenario and samples

* Added to readme

* Lint fix

* Linting

* Linting error fix

* Addressed comments and added extra properties

* Linting

* Fixing patch skip post action

* Linting

* Addressing comments

* Reverting patchDetails changes

* Fixing pipeline error

* Linting

* Removed invalid required tag

* Fixed examples

* Addressed comments

---------

Co-authored-by: Paul Dorsch <[email protected]>
Co-authored-by: Paul Dorsch <[email protected]>

* Update microsoft.app 2023-08-01-preview for new DaprSubscription API feature (Azure#25631)

* ACA Dapr Subscription: WIP

Signed-off-by: Bernd Verst <[email protected]>

* Add Dapr Subscription examples

Signed-off-by: Bernd Verst <[email protected]>

* complete examples

Signed-off-by: Bernd Verst <[email protected]>

* Add DaprSubscription descriptions

Signed-off-by: Bernd Verst <[email protected]>

* include updated readme

Signed-off-by: Bernd Verst <[email protected]>

* fix casing

Signed-off-by: Bernd Verst <[email protected]>

* formatting

Signed-off-by: Bernd Verst <[email protected]>

* Use Types v5

Signed-off-by: Bernd Verst <[email protected]>

* Revert "Use Types v5"

v5 causes several mysterious SDK build errors in this branch on this PR. To avoid this error we will stick to v3 for now. We will upgrade to v5 in a future net new branch.

This reverts commit 6a43aacf16132ff1918cbeb9d9b15c74b3a55a78.

---------

Signed-off-by: Bernd Verst <[email protected]>

* Support log streaming on build resource (Azure#25902)

* updated the build resource to support build upload and log streaming, with a token to be passed in via header

* add x-ms-secret and update buildendpoint -> endpoint

* retrieve log stream and upload endpoint tokens through post requests, as they are secrets

* remove secret flag from tokenendpoint property, as we expect users to retrieve it

* rename file

* updated format of how build object returns auth token

* fix example

* added expiration date to token response

* fix auth token to also include build resource information

* renamed getAuthToken -> listAuthToken and removed resource information from token response

* fix examples

* updated samples to be more accurate

* Update serviceBinds with clientType and customizedKeys (Azure#26111)

* Update serviceBinds with clientType and customizedKeys

* force ci

* fix missing }

---------

Co-authored-by: Ahmed ElSayed <[email protected]>

* Adds new Dapr Component Resiliency Feature to existing version `Release app microsoft.app 2023 08 01 preview` (Azure#26107)

* Add Dapr Component Resiliency Policies APIs to ContainerApps 2023-08-01-preview

Signed-off-by: Bernd Verst <[email protected]>

* Update deletion return code

* Fix variable name

* Remove invalid status code from example

* prettify

* Register new resource in readme

* Remove existing import cycle

* Use integers not integer strings

* Fix delete status codes

* More changes

* Update examples

* Revert "Remove existing import cycle" which causes unrelated failures

This reverts commit 8ce1a74f9d12b333c43c324f1b2250a64a0f1add.

---------

Signed-off-by: Bernd Verst <[email protected]>

* Add API spec for ContainerApp Resiliency (Azure#25732)

Signed-off-by: Hal Spang <[email protected]>

* Add extendedlocation for job (Azure#26196)

* Add extendedlocation for job

* Add sample for container apps on connectedEnvironment

* Fixes for examples

* Fix connected environment example

* Update old examples

* removing SourceToCloud patching from 08-01-preview (Azure#26245)

* Add diagnostic routes to Container App Jobs (Azure#26227)

* Add diagnostic routes to Container App Jobs

* Update

* Update

* Update

* Update

* Update

* Update

* Update operationIds

---------

Co-authored-by: Michimune Kohno <[email protected]>

* Update app resiliency rest api spec (Azure#26357)

Signed-off-by: Yash Nisar <[email protected]>

* Fix ACA component resiliency examples (Azure#26441)

* Fix arm review comments for container app (Azure#26455)

* update

* update

* update

* Fix SDK build error for Microsoft.App (Azure#26416)

* Fix SDK build error for Microsoft.App

* Update

* update

* update

* update

---------

Co-authored-by: Michimune Kohno <[email protected]>

* Fix arm review comments for container app (Azure#26479)

* fix

* update

* update

* Fix detectorProperties request path (Azure#26431)

* fix list exception

* change param to enum

* fix lint error

* revert not working changes

* fix enum

* rearrange

---------

Co-authored-by: Chenghui Yu <[email protected]>

* Enable dynamicJsonColumns for log analytics configuration (Azure#26432)

* Enable dynamicJsonColumn for log analysis configuration

* Add missed s

* Fix code style

* Update container app usages value to float (Azure#26448)

* update

* update

* update

* Fix App Resiliency retry example (Azure#26560)

Signed-off-by: Yash Nisar <[email protected]>

* Fix lint warning for container app (Azure#26523)

* fix warning

* fix warning

* upate

---------

Signed-off-by: Bernd Verst <[email protected]>
Signed-off-by: Hal Spang <[email protected]>
Signed-off-by: Yash Nisar <[email protected]>
Co-authored-by: zhenqxuMSFT <[email protected]>
Co-authored-by: yalixiang <[email protected]>
Co-authored-by: Paul Dorsch <[email protected]>
Co-authored-by: Michael Dai <[email protected]>
Co-authored-by: bgashirabake <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Harry Li <[email protected]>
Co-authored-by: Paul Dorsch <[email protected]>
Co-authored-by: Bernd Verst <[email protected]>
Co-authored-by: Ahmed ElSayed <[email protected]>
Co-authored-by: Ahmed ElSayed <[email protected]>
Co-authored-by: halspang <[email protected]>
Co-authored-by: LaylaLiu-gmail <[email protected]>
Co-authored-by: michimune <[email protected]>
Co-authored-by: Michimune Kohno <[email protected]>
Co-authored-by: Yash Nisar <[email protected]>
Co-authored-by: Seris370 <[email protected]>
Co-authored-by: Chenghui Yu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Azure.Core azure-core Client This issue points to a problem in the data-plane of the library. EngSys This issue is impacting the engineering system.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Pipeline to Validate Changes Made to code-quality-reports

3 participants