Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create custom-configuration.md #40967

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

mckennabarlow
Copy link

@mckennabarlow mckennabarlow commented May 16, 2024

Adding public documentation for how to provide custom config files to customize AppCAT code analysis

Summary

Fixes #Issue_Number (if available)


Internal previews

📄 File 🔗 Preview link
docs/azure/migration/appcat/custom-configuration.md docs/azure/migration/appcat/custom-configuration

Adding public documentation for how to provide custom config files to customize AppCAT code analysis
@mckennabarlow mckennabarlow self-assigned this May 16, 2024
@BillWagner BillWagner modified the milestones: May 2024, June 2024 Jun 7, 2024
@BillWagner BillWagner modified the milestones: June 2024, July 2024 Jul 8, 2024
@BillWagner
Copy link
Member

ping @alexwolfmsft @CamSoper for review

@CamSoper
Copy link
Contributor

CamSoper commented Aug 1, 2024

I'll give this a review Friday. Thanks, @mckennabarlow!

Copy link
Contributor

@CamSoper CamSoper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks great, @mckennabarlow! I made a few suggestions for readability. I'll also send a PR to this branch to add this doc to the TOC.

title: How to customize analysis with JSON run configurations
description: Learn how to include a JSON file to configure your code assessment
ms.topic: conceptual
ms.date: 5/16/2024
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ms.date: 5/16/2024
ms.date: 08/02/2024

ms.topic: conceptual
ms.date: 5/16/2024
author: mckennabarlow
ms.author: mckennabarlow
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ms.author: mckennabarlow
ms.author: mcbarlow

Use your MS alias for ms.author 🙂

Comment on lines +23 to +34
## How to provide run config

For CLI:

- in the interactive mode CLI will ask if you want to provide run config and then if you said yes, asks to type/paste path to the run config file.
- in non-interactive mode there is new argument `-c|--config` which allows to provide the path to the run config json.

For VS:

- at the step where we allow to edit analysis settings we added a text box and `Browse` button to let you specify a run config json.

After analysis is done, run config is stored in the report and re-used if user opens report again and refreshes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## How to provide run config
For CLI:
- in the interactive mode CLI will ask if you want to provide run config and then if you said yes, asks to type/paste path to the run config file.
- in non-interactive mode there is new argument `-c|--config` which allows to provide the path to the run config json.
For VS:
- at the step where we allow to edit analysis settings we added a text box and `Browse` button to let you specify a run config json.
After analysis is done, run config is stored in the report and re-used if user opens report again and refreshes.
## How to provide a JSON run configuration
### Using the CLI
- **Interactive mode**: The CLI will ask if you want to provide run configuration. Upon selecting *yes*, it prompts the user to type/paste the path to the run configuration file.
- **Non-interactive mode**: Use the `-c|--config` argument. This allows you to provide the path to the JSON run configuration.
### Using Visual Studio
- When editing analysis settings, use the UI to specify JSON run configuration.
After analysis is done, run configuration is stored in the report. The configuration is re-used if the user reopens the report and refreshes.

Comment on lines +13 to +21
Before running an analysis we discover all rules and analyzers provided to AppCAT and then use all that match current project's `traits`. However some results could be too noisy for some applications.

To control/scope rules and/or analyzers used, you can provide a json run config file which would allow you to:

- change default settings to include or exclude some binaries
- disable existing specific rules or specific analyzers
- change properties in existing rules
- add new rules
- add new analyzers for existing or new rules
Copy link
Contributor

@CamSoper CamSoper Aug 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Before running an analysis we discover all rules and analyzers provided to AppCAT and then use all that match current project's `traits`. However some results could be too noisy for some applications.
To control/scope rules and/or analyzers used, you can provide a json run config file which would allow you to:
- change default settings to include or exclude some binaries
- disable existing specific rules or specific analyzers
- change properties in existing rules
- add new rules
- add new analyzers for existing or new rules
Before running an analysis, AppCAT discovers all rules and analyzers provided to it and then uses that information to match the project's `traits`. However, the analysis results can be overly verbose when using the default configuration.
To control/scope rules and/or analyzers used, you can provide a JSON run configuration file. This allows you to:
- Change default settings to include or exclude some binaries.
- Disable existing specific rules or specific analyzers.
- Change properties in existing rules.
- Add new rules.
- Add new analyzers for existing or new rules.

Comment on lines +90 to +94
`Settings` section contains globing includes or excludes applied to filter in or out binaries that are going to be analyzed. It probably makes sense to run binary analysis once to see any red flags, however results for binaries are pretty verbose. Some findings could indeed flag a real problem inside binary dependency application is using without realizing it. However others could be (although valid) but in code paths not used by the application and could be ignored. To avoid being overwhelmed by amount of results for binaries, users could include only ones they really interested in instead of scanning all of them.

`Rules` section contains rules definitions which could override existing rules' properties, disable existing rule or even define new ones.

`Analyzers` section contains analyzers, which similarly to rules could be disabled or new analyzers could be added.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`Settings` section contains globing includes or excludes applied to filter in or out binaries that are going to be analyzed. It probably makes sense to run binary analysis once to see any red flags, however results for binaries are pretty verbose. Some findings could indeed flag a real problem inside binary dependency application is using without realizing it. However others could be (although valid) but in code paths not used by the application and could be ignored. To avoid being overwhelmed by amount of results for binaries, users could include only ones they really interested in instead of scanning all of them.
`Rules` section contains rules definitions which could override existing rules' properties, disable existing rule or even define new ones.
`Analyzers` section contains analyzers, which similarly to rules could be disabled or new analyzers could be added.
Looking at the JSON objects contained in the preceding example:
- `analysis.settings` contains global includes or excludes. These include and exclude binaries to be analyzed. It's recommended to run binary analysis once to see any red flags, but results for binaries tend to be verbose. Some of the results could flag problems inside dependencies your application is consuming. Other results could be valid problems in your app, but located in unused code paths. These can be ignored to reduce the verbosity of the results.
- `analysis.rules` contains rules definitions. These definitions override or disable existing rules, as well as define new rules.
- `analysis.analyzers` contains analyzers. Similarly to the rules definitions, these can be disabled, overridden, or added on to.

@BillWagner BillWagner modified the milestones: July 2024, August 2024 Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants