Skip to content

Add --exclude glob filter for CodeQL compiled-language false positives - #141

Merged
Malcolmnixon merged 2 commits into
mainfrom
feature/sarif-exclude-glob-filter
Sep 14, 2026
Merged

Add --exclude glob filter for CodeQL compiled-language false positives#141
Malcolmnixon merged 2 commits into
mainfrom
feature/sarif-exclude-glob-filter

Conversation

@Malcolmnixon

Copy link
Copy Markdown
Member

Summary

CodeQL's paths-ignore configuration only affects the extraction/build scope for compiled languages (C#, Java, C++, Go) — it does not filter the raw SARIF findings. Any file the compiler touches (including generated code under bin/obj) still appears in results regardless of config. Actions like advanced-security/filter-sarif exist to post-process SARIF and drop findings by glob pattern; this PR adds that capability natively to SarifMark so users don't need an extra workflow step.

Changes

  • New repeatable --exclude <glob> CLI option that filters SarifFinding entries by their Uri using Microsoft.Extensions.FileSystemGlobbing, applied after reading the SARIF file and before --enforce/--report processing (excluded findings don't affect the exit code or the generated report).
  • Findings with no physical location (null Uri) are never excluded.
  • Added Microsoft.Extensions.FileSystemGlobbing as a 12th OTS item, with design/verification/reqstream documentation and SysML2 model updates following the existing DemaConsulting.TestResults pattern (verified via SarifMark's own tests, no --validate hook).
  • Added feature requirements, design docs, and verification docs for the new option across the Cli, Cli.Context, Sarif, and Sarif.SarifResults units, plus a new system-level end-to-end integration test.
  • Updated README.md and the user guide (usage.md, faq.md) with --exclude documentation and a bin/obj CodeQL example.
  • Fixed two pre-existing documentation issues found during formal review: a README --help transcript inaccuracy (missing --depth range qualifier) and a misleading docs/verification/ots.md sentence about how several OTS tools are verified.

Example

sarifmark results.sarif --exclude "**/bin/**" --exclude "**/obj/**" --enforce --report report.md

Validation

  • pwsh ./build.ps1 — build succeeds, 0 warnings, 663/663 tests pass across net8.0/net9.0/net10.0.
  • pwsh ./fix.ps1 / pwsh ./lint.ps1 — clean (yamllint, cspell, markdownlint-cli2, dotnet format, sysml2tools lint).
  • dotnet reqstream --lint / --enforce --root-tags public,quality — no orphaned or unsatisfied requirements introduced by this change.
  • dotnet reviewmark --lint / --plan --enforce — full coverage for the new OTS item.
  • dotnet versionmark --lint — clean.
  • Built-in code-review agent: no significant issues found (two passes, including the follow-up fix round).
  • Formal reviews run against all 12 affected review-sets (Purpose, Decomposition, SarifMark-Architecture, SarifMark-Design, SarifMark-Verification, SarifMark-AllRequirements, SarifMark-Program, SarifMark-Cli, SarifMark-Cli-Context, SarifMark-Sarif, SarifMark-Sarif-SarifResults, OTS-FileSystemGlobbing) — all approved after fixing 5 Medium-severity documentation/traceability findings from the first pass.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

CodeQL's paths-ignore config only affects extraction/build scope for
compiled languages (C#, Java, C++, Go); findings from generated code in
bin/obj still appear in the raw SARIF output regardless of config. This
adds a repeatable --exclude <glob> CLI option so SarifMark can filter
findings by file-path glob pattern natively, without requiring an extra
Action/step (e.g. advanced-security/filter-sarif) in CI workflows.

- New repeatable --exclude <glob> option filters SarifFinding entries by
  Uri using Microsoft.Extensions.FileSystemGlobbing, applied after
  reading the SARIF file and before --enforce/--report processing.
- Findings with no physical location (null Uri) are never excluded.
- Added Microsoft.Extensions.FileSystemGlobbing as a 12th OTS item with
  full design/verification/reqstream documentation and SysML2 model
  updates, following the existing DemaConsulting.TestResults pattern.
- Added feature requirements, design docs, and verification docs for the
  new CLI option across the Cli, Cli.Context, Sarif, and Sarif.Results
  units, plus a new system-level end-to-end integration test.
- Updated README and user guide (usage.md, faq.md) with --exclude
  documentation and a CodeQL bin/obj example.
- Fixed a pre-existing README --help transcript inaccuracy (--depth
  range qualifier) and a misleading OTS verification-strategy sentence
  found during formal review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 14, 2026 03:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical CLI parsing issue and several requested test and documentation updates remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR adds repeatable --exclude <glob> filtering for SARIF findings before enforcement and reporting, with tests, dependency updates, documentation, and traceability artifacts.

Changes:

  • Added URI glob filtering via Microsoft.Extensions.FileSystemGlobbing.
  • Added unit, integration, and end-to-end coverage.
  • Updated user documentation, requirements, design, verification, SysML2, and review metadata.

Review findings:

  • Critical: Reject recognized options used as a missing --exclude value and add regression coverage.
  • Moderate: Add multi-run count, help, URI-format, and metadata test cases.
  • Nit: Align documented processing order and include SysML2Tools in the OTS verification list.
File summaries
File Summary
test/DemaConsulting.SarifMark.Tests/Sarif/SarifResultsTests.cs Tests filtering and metadata preservation.
test/DemaConsulting.SarifMark.Tests/ProgramTests.cs Tests filtering, reporting, and enforcement.
test/DemaConsulting.SarifMark.Tests/IntegrationTests.cs Tests compiled-language filtering end to end.
test/DemaConsulting.SarifMark.Tests/Cli/ContextTests.cs Tests exclusion context parsing.
test/DemaConsulting.SarifMark.Tests/Cli/CliTests.cs Tests CLI exclusion handling.
src/DemaConsulting.SarifMark/Sarif/SarifResults.cs Implements URI glob filtering.
src/DemaConsulting.SarifMark/Program.cs Applies filtering and reports results.
src/DemaConsulting.SarifMark/DemaConsulting.SarifMark.csproj Adds the globbing dependency.
src/DemaConsulting.SarifMark/Cli/Context.cs Parses repeatable exclusion patterns.
requirements.yaml Registers related requirements and OTS coverage.
README.md Documents the new option and usage.
docs/verification/sarifmark/sarif/sarif-results.md Defines SARIF-results verification scenarios.
docs/verification/sarifmark/sarif.md Documents SARIF verification.
docs/verification/sarifmark/program.md Documents program-level verification.
docs/verification/sarifmark/cli/context.md Documents context verification.
docs/verification/sarifmark/cli.md Documents CLI verification.
docs/verification/sarifmark.md Updates system verification coverage.
docs/verification/ots/filesystem-globbing.md Defines globbing dependency verification.
docs/verification/ots.md Updates the OTS verification strategy.
docs/user_guide/usage.md Adds exclusion usage guidance.
docs/user_guide/faq.md Explains CodeQL generated-code filtering.
docs/sysml2/model/sarifmark.sysml Adds the dependency to the system model.
docs/sysml2/model/ots.sysml Models the globbing dependency.
docs/reqstream/sarifmark/sarif/sarif-results.yaml Adds SARIF-results requirements.
docs/reqstream/sarifmark/sarif.yaml Adds SARIF filtering requirements.
docs/reqstream/sarifmark/program.yaml Adds program requirements.
docs/reqstream/sarifmark/cli/context.yaml Adds context requirements.
docs/reqstream/sarifmark/cli.yaml Adds CLI requirements.
docs/reqstream/sarifmark.yaml Adds system-level requirements.
docs/reqstream/ots/filesystem-globbing.yaml Defines dependency requirements.
docs/design/sarifmark/sarif/sarif-results.md Documents filtering design.
docs/design/sarifmark/sarif.md Documents SARIF integration.
docs/design/sarifmark/program.md Documents processing flow.
docs/design/sarifmark/cli/context.md Documents exclusion state.
docs/design/sarifmark/cli.md Documents the CLI contract.
docs/design/sarifmark.md Updates system design and data flow.
docs/design/ots/filesystem-globbing.md Documents globbing integration.
docs/design/ots.md Registers the new OTS dependency.
docs/design/introduction.md Updates the design index.
.reviewmark.yaml Adds OTS review coverage.
Review details

Suppressed comments (5)

docs/design/sarifmark.md:152

  • The sequence here still documents report generation before enforcement, but ProcessSarifAnalysis checks context.Enforce before writing the report. Please reorder these steps so the design flow matches the actual control flow.
7. If `--report` was supplied, the markdown string is written to the specified file with
   `File.WriteAllText`.
8. If `--enforce` is set and issues were found, `Context.WriteError` sets the exit code to 1.
9. `Program.Main` returns `Context.ExitCode` to the shell.

docs/verification/ots.md:9

  • The updated list of tools exposing --validate still omits SysML2Tools, although this document later describes a dotnet sysml2tools --validate CI step at lines 41-42. Include SysML2Tools here so the verification strategy is internally consistent.
1. **Self-validation output**: Tools that expose a `--validate` flag (BuildMark, FileAssert, ReqStream, ReviewMark,
   VersionMark) are exercised through their own `--validate` self-validation mechanism as separate CI pipeline
   steps; passing output confirms the tool is installed and all advertised features are operational.

src/DemaConsulting.SarifMark/Program.cs:237

  • The excluded-count summary is specified as the total removed across all runs, but the Program-level test only uses a single-run SARIF with one finding. Add a multi-run case with matches in more than one run to verify that this Sum-based count remains correct.
    src/DemaConsulting.SarifMark/Sarif/SarifResults.cs:437
  • This fixed-root call is documented as handling relative, file://, Unix-absolute, and Windows-drive URIs, but the tests only cover relative paths and a simple Unix-style file:/// URI. Add cases for absolute and Windows-drive forms (including separator/drive behavior), otherwise a platform-specific mismatch could silently leave CodeQL findings unfiltered.
    test/DemaConsulting.SarifMark.Tests/Sarif/SarifResultsTests.cs:1548
  • This test uses only one run even though its summary and the design contract say metadata is preserved for every run. A regression that drops or changes metadata on a later run would still pass; use a multi-run fixture and assert ToolName, ToolVersion, and FileCount for both runs (the filtering test does not check metadata).
  • Files reviewed: 40/40 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/DemaConsulting.SarifMark/Cli/Context.cs
Comment thread src/DemaConsulting.SarifMark/Program.cs
Copilot AI review requested due to automatic review settings September 14, 2026 03:42
- Fix GetRequiredStringArgument (shared by --log, --sarif, --report,
  --heading, --results, and --exclude) to reject a following token that
  is itself a recognized option, instead of silently consuming it as
  the value (e.g. --exclude --enforce no longer swallows --enforce).
- Apply the same fix to GetRequiredIntArgument (--depth/--report-depth)
  for structural consistency, using the shared KnownOptionTokens set.
- Add regression tests for --exclude, --sarif, and --depth followed by
  another option.
- Add an assertion to Program_Main_HelpFlag_DisplaysHelp confirming the
  --exclude help line is present.
- Link new test names to existing ReqStream requirements (no new
  requirement IDs).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No blocking issues were identified; only a minor README command-example correction was noted.

Review details

Suppressed comments (1)

README.md:140

  • The PR description's example uses sarifmark results.sarif ..., but this parser accepts the SARIF path only after --sarif; a positional results.sarif is rejected as an unsupported argument. Please update that example to match the runnable form shown here, e.g. sarifmark --sarif results.sarif ....
sarifmark --sarif analysis.sarif --report report.md --exclude "**/bin/**" --exclude "**/obj/**"
  • Files reviewed: 40/40 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@Malcolmnixon
Malcolmnixon merged commit 68e6c11 into main Sep 14, 2026
16 checks passed
@Malcolmnixon
Malcolmnixon deleted the feature/sarif-exclude-glob-filter branch September 14, 2026 03:56
This was referenced Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants