Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .reviewmark.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,13 @@ reviews:
- docs/design/ots/test-results.md
- docs/verification/ots/test-results.md

- id: OTS-FileSystemGlobbing
title: Review that FileSystemGlobbing Provides Required Functionality
paths:
- docs/reqstream/ots/filesystem-globbing.yaml
- docs/design/ots/filesystem-globbing.md
- docs/verification/ots/filesystem-globbing.md

# Shared Packages
- id: Shared-SarifMark
title: Review that Shared SarifMark Package Provides Required Functionality
Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,9 @@ Options:
--enforce Return non-zero exit code if issues found
--log <file> Write output to log file
--sarif <file> SARIF file to process
--exclude <glob> Exclude findings whose location matches glob (repeatable)
--report <file> Export analysis results to markdown file
--depth <depth> Markdown header depth for report (default: 1)
--depth <depth> Markdown header depth for report (1-6, default: 1)
--heading <text> Custom heading for report (default: [ToolName] Analysis)
```

Expand All @@ -133,6 +134,12 @@ sarifmark --sarif analysis.sarif --report report.md
sarifmark --sarif analysis.sarif --report report.md --heading "Code Quality Analysis"
```

**Exclude generated code from a report:**

```bash
sarifmark --sarif analysis.sarif --report report.md --exclude "**/bin/**" --exclude "**/obj/**"
```

**Enforce quality gate in CI/CD:**

```bash
Expand Down Expand Up @@ -273,3 +280,4 @@ SarifMark is built with the following open-source projects:
- [.NET](https://dotnet.microsoft.com/) - Cross-platform framework for building applications
- [SARIF](https://sarifweb.azurewebsites.net/) - Static Analysis Results Interchange Format specification
- [DemaConsulting.TestResults](https://github.com/demaconsulting/TestResults) - Test results parsing library
- [Microsoft.Extensions.FileSystemGlobbing](https://www.nuget.org/packages/Microsoft.Extensions.FileSystemGlobbing) - Glob pattern matching used by `--exclude`
6 changes: 3 additions & 3 deletions docs/design/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ Local items:

OTS items:

- **BuildMark**, **DemaConsulting.TestResults**, **FileAssert**, **Pandoc**, **ReqStream**, **ReviewMark**, **SonarMark**,
**SysML2Tools**, **VersionMark**, **WeasyPrint**, **xUnit v3**: integration and usage design for each OTS
software item used in the project pipeline.
- **BuildMark**, **DemaConsulting.TestResults**, **FileAssert**, **Microsoft.Extensions.FileSystemGlobbing**,
**Pandoc**, **ReqStream**, **ReviewMark**, **SonarMark**, **SysML2Tools**, **VersionMark**, **WeasyPrint**,
**xUnit v3**: integration and usage design for each OTS software item used in the project pipeline.

Shared packages:

Expand Down
17 changes: 12 additions & 5 deletions docs/design/ots.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# OTS Dependencies

SarifMark uses eleven OTS software items: ten DEMA Consulting pipeline tools and the
xUnit v3 testing framework. All eleven items are consumed as .NET tools or NuGet packages
and are managed through the local tool manifest and the project dependency lock files.
Per-item integration designs are documented in the `ots/` sub-folder.
SarifMark uses twelve OTS software items: ten DEMA Consulting pipeline tools, the
xUnit v3 testing framework, and Microsoft.Extensions.FileSystemGlobbing. All twelve items
are consumed as .NET tools or NuGet packages and are managed through the local tool
manifest and the project dependency lock files. Per-item integration designs are
documented in the `ots/` sub-folder.

## Selection Criteria

Expand All @@ -22,6 +23,11 @@ Testing framework selection (xUnit v3) is based on native TRX output support, co
with the VSTest adapter required by ReqStream, and the established ecosystem around xUnit
in the .NET community.

Microsoft.Extensions.FileSystemGlobbing is selected because it is the Microsoft-owned
globbing engine already used throughout the .NET ecosystem, giving users glob semantics
they are already familiar with and avoiding hand-rolled glob parsing logic within
SarifMark.

## Version Management Policy

OTS package versions are managed through Dependabot pull requests for NuGet and Node.js
Expand All @@ -39,7 +45,8 @@ versions are pinned in the local tool manifest.

All OTS items are consumed as CLI tools invoked from CI/CD pipeline scripts, as a NuGet
package referenced directly by the main project (DemaConsulting.TestResults, used by the
SelfTest subsystem to collect and serialize self-validation results), or as a NuGet package
SelfTest subsystem to collect and serialize self-validation results; Microsoft.Extensions.FileSystemGlobbing,
used by `SarifResults.Exclude` to filter findings by glob pattern), or as a NuGet package
referenced by the test project (xUnit v3). No wrapper classes are introduced at the
application level; tools are invoked directly via `dotnet tool run` or their shell
command, and packages are referenced through standard NuGet project references.
Expand Down
58 changes: 58 additions & 0 deletions docs/design/ots/filesystem-globbing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
## Microsoft.Extensions.FileSystemGlobbing

### Purpose

`Microsoft.Extensions.FileSystemGlobbing` is a NuGet package produced by Microsoft. It provides a
glob-pattern matching engine (`Matcher`) that determines whether a candidate path matches one or
more include patterns (including `*`, `?`, and recursive `**` wildcards). SarifMark uses it to
implement the `--exclude` CLI option, which removes SARIF findings whose `Uri` matches a
user-supplied glob pattern before enforcement and report generation.

The package was chosen because it is the Microsoft-owned globbing engine already used throughout
the .NET ecosystem (MSBuild item globs, `dotnet watch`, ASP.NET Core static file providers), which
avoids hand-rolling glob parsing and matching logic and gives users glob semantics they are
already familiar with.

### Classification

`Microsoft.Extensions.FileSystemGlobbing` is produced and released independently by Microsoft as
part of the `Microsoft.Extensions` family of packages. It is not part of the SarifMark product and
its internal design requirements do not drive SarifMark requirements. It is therefore classified
as an OTS software item.

### Features Used

- **`Matcher`** — the matching engine. One instance is constructed per call to
`SarifResults.Exclude`, with one `AddInclude(pattern)` call per supplied `--exclude` pattern.
- **`AddInclude(string pattern)`** — registers a glob pattern that a candidate path must match.
- **`MatcherExtensions.Match(this Matcher, string searchDirectory, IEnumerable<string> files)`** —
the in-memory matching overload used to test a single candidate `Uri` against the registered
patterns. This overload never touches the real filesystem; it evaluates the supplied strings
directly, so SARIF `Uri` values do not need to correspond to files that exist on disk.

### Integration Pattern

`Microsoft.Extensions.FileSystemGlobbing` is referenced as a `PackageReference` in the production
project `src/DemaConsulting.SarifMark/DemaConsulting.SarifMark.csproj`. No initialization or
configuration is required; the `Matcher` type is used directly.

Inside `SarifResults.Exclude`, a single `Matcher` is constructed and one `AddInclude` call is made
per pattern in the supplied glob list. Each finding's non-null `Uri` is then tested individually
against the matcher using a fixed root of `"/"` (via `matcher.Match("/", [uri]).HasMatches`) —
this root value was chosen because it was empirically observed to consistently match relative
paths, `file://` URIs, Unix absolute paths, and Windows drive-letter absolute paths (with either
`/` or `\` separators), whereas the no-root overload resolves paths relative to the current
working directory and silently fails to match absolute paths outside it. Findings are matched one
at a time (rather than batched) because batch matching was observed to mangle Windows
drive-letter prefixes in the returned `Path`/`Stem` values, making result correlation unreliable.
Findings with a `null` `Uri` are always retained and are never passed to the matcher. Matching is
case-insensitive by default (the default `Matcher()` constructor does not specify a
`StringComparison`), which was confirmed during implementation and is documented as the observed
behavior rather than assumed.

`Program.ProcessSarifAnalysis` invokes `SarifResults.Exclude` only when `context.ExcludeGlobs` is
non-empty. No error handling is required beyond what `Matcher`/`AddInclude` already provide;
malformed glob patterns do not throw during registration, and unmatched patterns simply result in
no findings being excluded.

There are no global initialization, thread-affinity, or disposal requirements.
16 changes: 11 additions & 5 deletions docs/design/sarifmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ provides path-safety helpers shared across the system.
- *Type*: CLI
- *Role*: Provider (the tool accepts arguments from the shell)
- *Contract*: Accepts flags and parameters (`--sarif`, `--report`, `--depth`, `--heading`,
`--validate`, `--results`, `--enforce`, `--log`, `--silent`, `--version`, `--help`).
`--exclude`, `--validate`, `--results`, `--enforce`, `--log`, `--silent`, `--version`, `--help`).
`--exclude <glob>` is repeatable and removes findings whose `Uri` matches the supplied
glob pattern before enforcement and report generation.
`--report-depth` is a **deprecated** alias for `--depth` and `--result` is a **deprecated** alias
for `--results`; both are accepted identically to their canonical forms but are intentionally
omitted from the `--help` output.
Expand Down Expand Up @@ -122,6 +124,8 @@ provides path-safety helpers shared across the system.
see *WeasyPrint Integration Design*
- **DemaConsulting.TestResults**: the OTS package used by the self-validation subsystem to collect, format, and
serialize test results — see *TestResults Integration Design*
- **Microsoft.Extensions.FileSystemGlobbing**: the OTS package used by `SarifResults.Exclude` to match finding
`Uri` values against user-supplied `--exclude` glob patterns — see *FileSystemGlobbing Integration Design*
- **SarifMark**: a released version of SarifMark itself, invoked as a shared package in the
CI pipeline to generate the CodeQL quality report — see *SarifMark Shared Package Integration Design*

Expand All @@ -139,11 +143,13 @@ The primary analysis data flow from SARIF input to markdown output:
4. In analysis mode, `SarifResults.Read` validates the file path, parses the JSON, validates
the SARIF structure, and constructs an immutable graph of `SarifRun` and `SarifFinding`
records.
5. `SarifResults.ToMarkdown` traverses the record graph and produces a UTF-8 markdown string.
6. If `--report` was supplied, the markdown string is written to the specified file with
5. If `--exclude` glob patterns were supplied, `SarifResults.Exclude` removes findings whose
`Uri` matches any of the patterns before enforcement or report generation.
6. `SarifResults.ToMarkdown` traverses the record graph and produces a UTF-8 markdown string.
7. If `--report` was supplied, the markdown string is written to the specified file with
`File.WriteAllText`.
7. If `--enforce` is set and issues were found, `Context.WriteError` sets the exit code to 1.
8. `Program.Main` returns `Context.ExitCode` to the shell.
8. If `--enforce` is set and issues were found, `Context.WriteError` sets the exit code to 1.
9. `Program.Main` returns `Context.ExitCode` to the shell.

The self-validation flow is a separate path in step 3 where `Validation.Run` exercises the
analysis flow end-to-end using a mock SARIF file and verifies the output.
Expand Down
3 changes: 2 additions & 1 deletion docs/design/sarifmark/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ command-line arguments.
- *Type*: In-process .NET instance properties
- *Role*: Provider
- *Contract*: Exposes `Version`, `Help`, `Silent`, `Validate`, `Enforce` (bool);
`SarifFile`, `ReportFile`, `Heading`, `ResultsFile` (string?); `Depth` (int); `ExitCode` (int).
`SarifFile`, `ReportFile`, `Heading`, `ResultsFile` (string?); `Depth` (int);
`ExcludeGlobs` (`IReadOnlyList<string>`, default empty); `ExitCode` (int).
All values are set during `Create` and are immutable after construction.
- *Constraints*: Properties are read-only after construction; `Depth` must be a positive
integer supplied after `--depth` (or legacy `--report-depth`).
Expand Down
10 changes: 9 additions & 1 deletion docs/design/sarifmark/cli/context.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@ When null, the report heading defaults to `"[ToolName] Analysis"`.
deprecated alias `--result`; `null` when not provided. The deprecated alias is accepted for
backwards compatibility but is intentionally omitted from `--help` output.

**ExcludeGlobs**: `IReadOnlyList<string>` — File-glob exclusion patterns supplied via one or
more `--exclude` parameters; default empty list. One entry is appended per occurrence of the
flag. Consumed by `SarifResults.Exclude` to remove matching findings before enforcement and
report generation.

**ExitCode**: `int` — Returns `0` until `WriteError` is called; returns `1` thereafter.
Derived from the internal `_hasErrors` flag.

Expand All @@ -55,7 +60,8 @@ Derived from the internal `_hasErrors` flag.
- *Returns*: `Context` — fully initialized instance
- *Preconditions*: `args` is not null.
- *Postconditions*: All properties are set from `args`; if `--log` was specified the
log `StreamWriter` is open and `AutoFlush` is `true`.
log `StreamWriter` is open and `AutoFlush` is `true`; `ExcludeGlobs` contains one entry
per `--exclude` occurrence, in the order supplied.

`Create` validates that `args` is non-null, constructs an `ArgumentParser`, calls
`ParseArguments`, copies parsed values into the new `Context` via `init`-only setters,
Expand Down Expand Up @@ -89,6 +95,8 @@ and calls `OpenLogFile` when a log path was specified.

`Create` throws `ArgumentException` for unrecognized tokens and for malformed value-bearing
flags (e.g., `--depth` not followed by an integer between 1 and 6 inclusive, or a string flag at end of args).
This includes `--exclude`, which throws `ArgumentException` when supplied without a value,
identically to the other value-bearing string flags.
It throws `InvalidOperationException` if the log file cannot be opened. `ArgumentNullException`
is thrown immediately if `args` is null. These exceptions propagate to `Program.Main`, which
translates them to exit code 1.
Expand Down
15 changes: 10 additions & 5 deletions docs/design/sarifmark/program.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,18 @@ then always print the banner; help flag → print help and return; validate flag
- *Returns*: `void`
- *Preconditions*: `context.Version`, `context.Help`, and `context.Validate` are all false.
- *Postconditions*: If `context.SarifFile` is a valid path to an existing SARIF file, the
results have been reported to the context output; if `context.ReportFile` is set, the
markdown report has been written to disk.
results have been reported to the context output; if `context.ExcludeGlobs` is non-empty,
matching findings have been removed and an excluded-count summary line has been reported
to the context output; if `context.ReportFile` is set, the markdown report has been
written to disk.

The method validates that `context.SarifFile` is non-null and non-whitespace; calls
`SarifResults.Read`; checks `context.Enforce` against `sarifResults.HasIssues`; and
conditionally writes the markdown report using `sarifResults.ToMarkdown` and
`File.WriteAllText`.
`SarifResults.Read`; reports the tool name, version, and result count; if
`context.ExcludeGlobs` is non-empty, calls `sarifResults.Exclude(context.ExcludeGlobs)` and
writes an `"Excluded {N} finding(s) matching --exclude patterns."` summary line where `N` is
the number of findings removed across all runs; checks `context.Enforce` against
`sarifResults.HasIssues`; and conditionally writes the markdown report using
`sarifResults.ToMarkdown` and `File.WriteAllText`.

### Error Handling

Expand Down
13 changes: 12 additions & 1 deletion docs/design/sarifmark/sarif.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,15 @@ contains three units:
- *Contract*: Returns `true` if any run contains at least one result.
- *Constraints*: None.

**SarifResults.Exclude**: Removes findings whose `Uri` matches a user-supplied glob pattern.

- *Type*: In-process .NET instance method
- *Role*: Provider
- *Contract*: Accepts `IReadOnlyList<string>? globPatterns`; returns a new `SarifResults`
with matching findings removed from every run. Findings with a `null` `Uri` are always
retained. Returns the same instance unchanged when `globPatterns` is null or empty.
- *Constraints*: None; malformed patterns simply match nothing.

### Design

The SARIF reading and reporting pipeline flows through the three units in sequence:
Expand All @@ -64,5 +73,7 @@ The SARIF reading and reporting pipeline flows through the three units in sequen
produce a `SarifRun` record. `ParseResults` uses `IsSuppressed` to filter suppressed
results before constructing `SarifFinding` records.
5. The completed `SarifResults` record is returned to `Program`.
6. `Program` calls `SarifResults.ToMarkdown` when `--report` is specified; the markdown
6. If `--exclude` glob patterns were supplied, `Program` calls `SarifResults.Exclude` to
remove matching findings from every run before enforcement or report generation.
7. `Program` calls `SarifResults.ToMarkdown` when `--report` is specified; the markdown
string is written to disk with `File.WriteAllText`.
29 changes: 28 additions & 1 deletion docs/design/sarifmark/sarif/sarif-results.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,27 @@ are skipped.
**ExtractFileCount**: Returns the length of the `artifacts` array in the run element; `0`
when the array is absent or not an array.

**Exclude**: Removes findings whose `Uri` matches a user-supplied glob pattern.

- *Parameters*: `IReadOnlyList<string>? globPatterns` — glob patterns to match against each
finding's `Uri`
- *Returns*: `SarifResults` — a new instance with matching findings removed from every run
- *Preconditions*: None.
- *Postconditions*: Returns `this` unchanged when `globPatterns` is null or empty. Otherwise
returns a new `SarifResults` in which each `SarifRun` retains only findings whose `Uri` is
`null`, or whose `Uri` does not match any supplied pattern. Each new `SarifRun` preserves
the original `ToolName`, `ToolVersion`, and `FileCount`.

`Exclude` builds a single `Microsoft.Extensions.FileSystemGlobbing.Matcher`, registering one
`AddInclude` call per supplied pattern. For each run, it filters `Results` by testing each
non-null `Uri` individually against the matcher using a fixed search root of `"/"` (chosen
because it was found to consistently match relative paths, `file://` URIs, and both Unix and
Windows absolute paths — the no-root overload instead resolves relative to the current
working directory and silently fails to match absolute paths outside it). Findings with a
`null` `Uri` are always retained and never passed to the matcher. Each filtered run is
reconstructed via the internal `SarifRun` constructor, preserving `ToolName`, `ToolVersion`,
and `FileCount` from the original run.

#### Error Handling

`Read` throws `ArgumentException` when `filePath` is null, empty, or whitespace.
Expand All @@ -79,6 +100,9 @@ violations (missing `version`, missing or empty `runs`, missing `tool` or `drive

`ToMarkdown` throws `ArgumentOutOfRangeException` when `depth` is outside `[1, 6]`.

`Exclude` does not throw for malformed glob patterns; `Matcher.AddInclude` does not validate
pattern syntax and a pattern that matches nothing simply excludes no findings.

The project file includes `<InternalsVisibleTo Include="DemaConsulting.SarifMark.Tests" />`
to allow the test assembly to construct instances directly for unit testing.

Expand All @@ -87,10 +111,13 @@ to allow the test assembly to construct instances directly for unit testing.
- **SarifRun** — each element of `Runs` is a `SarifRun` instance produced during `Read`.
- **SarifFinding** — each run's results contain `SarifFinding` instances.
- **System.Text.Json** — `JsonDocument.Parse` is used for all JSON parsing.
- **Microsoft.Extensions.FileSystemGlobbing** — `Matcher` is used by `Exclude` to match
finding `Uri` values against glob patterns — see *FileSystemGlobbing Integration Design*.

#### Callers

- **Program** — calls `SarifResults.Read(context.SarifFile)` and
- **Program** — calls `SarifResults.Read(context.SarifFile)`,
`sarifResults.Exclude(context.ExcludeGlobs)` (when `ExcludeGlobs` is non-empty), and
`sarifResults.ToMarkdown(depth, heading)` from `ProcessSarifAnalysis`.
- **Validation** — calls `SarifResults.Read` indirectly via `Program.Run` during
self-validation tests.
Loading